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ABSTRACT 

The report describes the Appalachian College 
Assessment Project, which developed a consortium to assess 
cooperatively the outcomes of the general education programs of a 
group of private liberal arts colleges in central Appalachia. 
Evaluation methods included a freshman/senior essay, the Academic 
Profile, the Pace Colleae Student Experiences Questionnaire, and 
senior interviews. The structured group interview process, developed 
in the consortium, has proven to be the most productive of the 
assessment methods. The senior interview involves one faculty member 
and several students for an intensive 2-hour conversation and has 
been highly rated by both students and faculty. Over the 3 project 
years, approximately 250 faculty and about 1260 students have 
participated in the interviews. Interviews were analyzed in relation 
to college goals, and each college was provided a written analysis of 
interview results. A series of conferences on assessment and general 
education provided faculty development as well as consortium 
direction. The consortium also provided a meaningful comparison group 
of similar institutions for individual colleges. The bulk of the 
document is made up of five appendixes which provide: the guidelines 
used in conducting the interviews; a sample transcript and analysis; 
guidelines for the freshman-senior essays; information on the 
Assessment Institute; and information on the assessment consortium 
conferences. (DB) 
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FINAL REPORT 



Project Suiranary 

The Appalachian College Assessment Project developed a consortium to 
assess cooperatively ^he outcomes of the general education programs of a group 
of private liberal arts colleges in central Appalachian The consortium 
eventually involved fourteen colleges which serve as a reference group for 
each other. Its multiple methods included a freshman/senior essay « the 
Academic Profile, the Pace C3EQ» and senior interviews. The interviews were 
developed in the consortiuiu and have proven to be the most productive of the 
assessment methods used and have been adopted by several other colleges and 
universities* A series of faculty conferences on assessment and general 
education were important in making the consortium a reality and as a form of 
faculty development. The project has become the basis for a slightly revised 
consortium which is funded by the colleges themselves. 
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Executive Summary 



Title: APPAIACHIAN COLLEGE ASSESSMENT CONSORTIUN 

Address: Faculty Scholars Program 
110 Maxwelton Court 
University of Kentucky 
Lexington, KY 40506--0347 

Contact: Karen Carey (606) 257-1202 
or 

Stacey Street (606) 257-3748 



Project Overview 

A consortium of fifteen private liberal arts colleges in Central 
Appalachia was formed in conjunction with the University of Kentucky's 
Department of Educational Policy Studies and its Appalachian College Program* 
The colleges involved were, in Kentucky: Berea, Union and Lindsey-Wilson; in 
North Carolina: Mars Hillf and Warren Wilson; in Tennessee: Maryville, 
Carson-Newman 9 King, and Milligan; in Virginia: Emery and Henry, Ferrum, 
Shenandoah, and Virginia Intermont; and in West Virginia, Wheeling Jesuit 
College. 

The original motivation for the consortium was to develop a cooperative 
approach to assessment that would be meaningful to the colleges and help them 
address the Southern Association's new Institutional Effectiveness criteria. 
The project involved three major efforts: organizational efforts related to 
building the consortium, developing the consortium's assessment approach, 
which also became a kind of faculty develoment effort, and activities related 
to collecting and analyzing data and feeding it back to its sources. The 
project developed a multiple-measures approach to assessing general education 
at the program level, and in the process developed a structured interview 
process which has been adopted by several other institutions. It is 
continuing with funding from the colleges and direction from a faculty member 
at one of the member colleges. 

Purpose 

The purpose of the project was to build a consortium that would become 
self-sustaining because it provided meaningful approaches to assessment which 
also would satisfy the institutional effectiveness criteria for accreditation 
by the Southern Association of Colleges and Schools. 

The culture of the colleges is unique in some ways. All are private, 
church-related liberal arts colleges in Appalachia with similar goals and 
purposes, quality faculty dedicated to teaching, similar student bodies, and 
real resource constraints due partly to their geographical isolation. We 
wanted to use or develop assessment methods appropriate to their curricula, 
missions and culture. The colleges also wanted to serve as a reference group 
or norm group for each other in interpreting assessment data. 

We wanted the actual process of participation in the consortium, as well 
as the assessment data, to help the colleges identify both strengths and 
weaknesses. In that sense we saw the project as a potential source of faculty 
renewal. We also assumed that some economies of scale would be inherent in 



the cooperative approach general ly, and by sharing the expertise available 
through the University of Kentucky in particular. 

Background and origins 

The Assessment Consortium grew out of an existing group of 38 Appalachian 
Colleges which participate in the UK Faculty Scholars/Appalachian College 
Program. Although those colleges had a history of connectedness through the 
University of Kentucky, the consortium required a major change in the level of 
commitment and the nature of their interactions. From the University's point 
of view, this was the first attempt to get the colleges working together as 
whole institutions, rather than serving the colleges by supporting individual 
faculty. From the colleges' viewpoint, this was the first time they were 
required to be involved in the planning and to make a financial contribution 
to the group effort. 

Project Description and Results 

The consortium was coordinated by a consortium director at the University 
of Kentucky. It started with five colleges, one from each of five central 
Appalachian states; by the end of the project fourteen colleges were members. 
The director served as a consultant, travelling several times a year to each 
college, working with faculty and deans, planning and holding conferences for 
all consortium faculty, collecting data, and coordinating the devleoproent of a 
multiple method approach to assessing general education at the program level. 

One zTiajor aspect of the project involved creating the consortium, so that 
both the deans and the faculty identified themselves as members and 
experienced their peers from the other institutions as colleagues. The 
primary means for accomplishing this was a series of conferences relating to 
assessment of some aspect of general education. About 225 faculty and 14 
deans were involved in the conferences, and both deans and faculty reported 
the ^conferences were among the most valuable aspects of the consortium. 
Another is the cross-pollination function served by providing information and 
ideas on assessment and other issues, either in person or in newsletters, from 
the consortium office. The director had to become familiar with the climate, 
processes and concerns of each college and to be able to provide a larger 
higher education context as a basis for understanding and planning assessment 
activities for the consortium. 

A second major effort was developing the actual multiple method approach 
to assessing general education. The consortium used the College Student 
Experiences Questionnaire, the Academic Frofilet a structured senior interview 
process, and a freshman-senior essay in attempting to assess the quality of 
their overall general education programs. The interview assessment has been 
the most valuable of these. Its value lies both in the process and the 
results. Both students and faculty value the experience. Seniors appreciate 
the fact that faculty are really listening for several hours. Combining 
several students and one faculty person in one interview group allows for an 
intensive two-hour conversation, during which studonts usually synthesize some 
of their experiences for the first time. Each faculty member conducts only 
one interview 9 so that the process informs a critical mass and provides a 
community of understanding on each campus. Faculty value the process because 
they trust it as a source of meaningful information which is clearly related 
to the curriculum and environment they have helped to create. The personal 
interaction in itself seems to be worthwhile for both. 



The interview questions were selected by the colleges through a travelling 
Delphi technique from a list developed by outstanding educators across the 
c^uStJy for tSe purpose of asseflsing general education. R*ther than asking 
stSenL to report on or give opinions about their general •<i^cation the 
incerviews require seniors to demonstrate their general education in « 
™sationa? fonnat. and in so doing become a very ^^^^^^J^^-f^'^P^f ^J^^, 
source of data. Over the three project years at least 250 faculty and about 
UeS students ierrmvolved in the interviews. The conversations were taped, 
transcribed and I^alyzed in relation to the goals of the college for general 
education by the director cf the consortium. A written f .J"^^^ 
for each college along with a personal presentation to the faculty with some 
in^erpretafiin'of the'coUege in comparison to the others. 
data indicate that the college is accomplishing itfi goals quite effectively, 
^^d in a way that reflects the philosophy of the institution, i-e-. through a 
strongly historical frame of reference, or with an emphasis social 
responsibility. Several colleges have modified courses or portions of their 
prog™s, retrained faculty or undertaken reviews of the general education 
portion of their curriculum based on the interview data. 

The third major effort was coordination and collection of data lot the 
thirteen colleges. The directors served as institutional " 
consortium as a whole and as institutional research advisors for individual 
coUeges. They developed group norms, analyzed in more depth the data from 
thP Academic Profile, and produced a combined report of the Pace College 
S^udenfSperiences Questionnaire data. Other administrative ta.Vs inc uded 
negotiatSg with ETS and with UCLA a.out group rates and service., and billing 
and accounting. The Academic Profile has been only marginally useful. 
p^oviSing Ut?le new information, at fairly high cost. The CSEQ provides a 
brolder context for interpretation of the interview data, both for the 
consortium as a whole and for the individual colleges. Only students 
wer- interviewed in each college during a given year, so the CSEQ helped 
indicate whether comments in the interviews were patterned or idiosyncratic, 
and provided useful information about many aspects of the 1«8%«^P^^^^"",- 

Evaluation was and is an ongoing aspeact of the consortium and each of its 
activUi^. It has been primarily conversational and formative. An Important 
Indicator if success is the continuation of the consortium with funding from 
the colleges. A more stable financial base is still needed for the consortium 
to actua!f.e its potential, however. Dissemination of 

project and its components will occur at a major conference to be held in 
Lexington in Fall 1990. Articles, professional presentations. «jd 
consultation with other institutions are also part of the dissemination 
process Information about the consortium process and the interview process 
has been requested by hundreds of institutions. The interviews have been 
adapted b^several large universities as well as other private liberal arts 
colleges and community colleges. 

Sunmarr and Conclusions . 

Developing a consortium of colleges is a very challenging and complex 
process wilh real potential for Improving the quality of "«'*"8"^r^^;. ^ 
education in these colleges. Because of its complexity, ^^^^ance planning 
should be done thoroughly and at every level. Both the process and the 
outcomes are well worthwhile I 
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THE APPALACHIAN COLLEGE ASSESSMENT CONSORTIUM 



Project Overview 

The consortium began with five colleges « one in each of the five Central 
Appalachian States. Though each college is tinique* they share similar 
missions 9 curricula and philosophies. All are private^ church-related liberal 
arts colleges « whose histories are related Lo their Appalachian location. 
They share similar strengths: dedicated faculty, commitment to quality 
undergraduate education, an emphasis on individual student development — and 
similar weaknesses: resource constraints, invisibility on the national higher 
education landscape, and the ensuing difficulty in recruiting faculty and 
studentF. Because of these commonalities, we believed that a consortial 
approach to institutional research and assessment could provide information 
which could help the colleges document their successes and plan to improve 
their effectiveness as institutions of higher education. 

The original five colleges were chosen by the Director of the University 
of Kentucky *d Appalachian College Program, primarily on the basis of her 
previous work with the deans of the colleges and their mutual estimation of 
the collegers ability to benefit from and contribute to a consortial effort. 
The five colleges were Berea, (Kentucky); Marjrville, (Tennessee); Wheeling 
Jesuit (Wes^ Virginia); Mars Hill, (North Carolina) and Emory and Henry, 
(Virginia). In the second year, eight colleges joined the consortium. During 
the third year, two dropped out and one joined. Altogether fourteen colleges 
were involved. The Consortium with ten colleges continues under the direction 
of a faculty member from Mars Hill College. 

The Appalachian College Assessment Consortitmi was started as a FIPSE 



project. It was intended partly to change the nature of the University's 
involvement with private colleges in the region by applying some of the 
resources of the University's Department of Educational Policy Studies and 
Evaluation to helping private liberal arts colleges address the new assessment 
criteria of the Southern Association. It was also intended to change the 
nature of the relationships of the colleges to each other. At the same time, 
we intended to develop cooperative approaches to assessment that would meet 
certain criteria: they had to 1) be meaningful to faculty 2) be valuable to 
students, 3) provide better information than the colleges could develop alone, 
U) serve as f. basis for improving the quality of the educational experience in 
the member colleges singly and as a group, and 5) be economically feasible. 

During the three project years the consortium made substantial progress 
towards its goals. A multiple-method approach to assessing general education 
evolved. It included an assessment institute and assessment conferences for 
faculty, senior interviews, freshman-senior essays, the Academic Profile, and 
the Pace "College Student Experiences Questionnaire." 

One of the most important products of the consortium was the development 
of the senior interviews. They were important for several reasons. First, 
the quality of the information the interviews yield is not available from any 
other source. It is rich, dense and context-specific. The interviews not 
only address questions of meaning, but they are consistent with the oral 
culture of Appalachia and the personal aspects of education necessary to 
creating the sense of community experienced in these colleges. The data from 
the interview transcripts also provide a context for the interpretation of 
other assessment data. Interviews make sense to faculty. Their face validity 
is helpful from an administrative viewpoint because it engages faculty in 
assessment issues within a concrete context and serves as a springboard for 
discussion of the meaning of other kinds of assessment data as well. 



activities and to accept other kinds of data collected as part of an 
assessment program. The fact that the interview process itself is educational 
is also important. It is valued by both faculty and students; in fact, the 
overall process became a form of faculty development. 

Although we saw the interviews as being particularly suited to piivite 
liberal arts colleges in Appalachia, both the method and the questions have 
been adopted by other institutions t including research universities, regional 
universities t community colleges and professional schools. While much work 
remains to be done in refining the analysis process, an important foundation 
has been established. 

The extent to which the success of the project depended on active 
consortium-building as an intentional part of the project had not been 
anticipated in the original plans. We learned that the Director has to serve 
an integrative function, devoting consistent effort to building and 
maintaining the consortium. Otherwise* efforts tend to concentrate on 
activities closer to home and to individual college interests. Consortium 
building had to occur both at the level of the deans and at the level of the 
faculty, so another major activity of the project involved creating 
opportunities for the college faculties to discuss assessment questions and 
project data together. A two-week institute on assessment was held the first 
year of the project, and each succeeding year two conferences on assessing 
some aspect of general education involved various groups of faculty. The 
ck -iferences were evaluated very positively by partipating deans and faculty 
who were stimulated to think about assessment in new ways and to interact with 
colleagues and exchange ideas and information about general education and what 
kinds of approaches have been effective. Sharing their own knowledge and 
benefitting from the experiences and knowledge of colleagues in the other 
colleges became a form of faculty development in its own right, in addition to 



the benefits to the asseesment programs. 

Perhaps more than any other type of higher education institution » the 
small private liberal arts colleges understand community. The Appalachian 
College Assessment Consortium has become the foundation for a new community. 
Funded by the member colleges « the consort iimi is continuing under the 
direction of a faculty member from Mars Hill College^ and is in a transitional 
stage. During its initial stage t some of the consortial relationships were 
intentionally left undefined, so that they could develop naturally, and also 
to prevent any sense of domination of the consortium by the University. 
Within the next year certain procedures will have to be defined and more 
clearly structured* The consortium has the potential to help the colleges 
expand their assessment scope, perhaps to pursue creative, cooperative 
approaches to assessing outcomes of their academic majors and student affairs 
programs. In so doing it can also serve as an ongoing center for faculty 
exchange and development that helps overcome some of the effects of geographic 
and academic isolation. 
Purpose 

Although we originally had defined the purposes of the consortium as 
developing meaningful, cooperative approaches to assessment, in retrospect, 
there were several purposes: 

1) We wanted to develop a consortium of colleges which would become self 
sustaining. Each of the consort itmi colleges was also a part of the 
University's Appalachian College Program, essentially a loosely knit voluntary 
group of 38 colleges whose deans attended an annual meeting at the University 
of Kentucky, and whose faculty were eligible for fellowships in the sciences 
and the humanities to enable them to conduct research at the University of 
Kentucky. There is no cost to the colleges for participation* The Program 
has been funded mainly through foundations and was designed to help the 
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institutions by helping individual faculty. The assessment consortium was 
quite different in concept. Rather than benefitting individual faculty 
members with fellowships, the consortium was planned to engage the entire 
college in a mutually beneficial cooperative effort, with each college 
contributing financially and practically to its success. 

2. A major purpose of the project was to assist the colleges, most of 
which, as members of the Southern Association, were trying to meet the new 
Institutional Effectiveness Criterion for accreditation, by serving as an 
identifiable institutional research ftanction. Most did not have the resources 
to hire a director of institutional research. Indeed, if the function existed 
at all it was usually a part of one faculty person's many responsibilities. 

In the process of meeting the new Criteria we also wanted to provide better 
information for improving the quality of the educational experience at the 
colleges than would otherwise be available. The project also seemed to 
present an opportunity for the colleges to meet the new criteria and, as 
relatively ••invisible*' irstitutions, to document their cuccesses as a group. 

3. There are some instances in which a national norming Is not a 
meaningful comparison group. The colleges as a group are unique in many ways, 
and developing ways these similar institutions could serve as a reference or 
crmparison group for each other as they assessed the effectiveness of their 
general education programs was another purpose of the project. 

A, Just as the composition of the reference group was important, the 
assessment methods used had to be appropriate to these colleges and their 
curricula. Another purpose of the consortium was to develop such assessment 
methods. 

5. Providing a needed service at less cost than otherwise possible was 
another goal. One co-diiector of the project was an institutional researcher 
who, along with a faculty member from the Department of Educational Policy 




studies and Evaluation at the University of Kentucky* could conduct in-depth 
analyses of the group data and help generate reports for the colleges which 
would enable them to improve their academic programs* 

Originally it was assumed that saving money was at least a goal, if not a 
purpose, of the consortium. While it operates in a manner designed to 
minimize costs and maximize benefits, it did in fact cost the institutions 
more than they were previously spending on assessment, but less than they 
would have spent collectively. While there are economic advantages to a 
consortium, there are also administrative costs* The benefits of the 
consortium derive partly from the process of cooperation itself, and partly 
from the quality of the results. 

The original plan involved hiring one institutional research person to 
work with five colleges to develop the consortium and a cooperative approach 
to assessment. Each year the group would add five to ten more colleges, with 
the idea that by the end of the three years the group could be self 
supporting. Participation for the first year was essentially underwritten by 
FIPSE, and each succeeding year colleges were to pay a membership fee* The 
exact mechanism for this arrangement was not clearly thought out at the 
beginning, and it proved to be quite cumbersome* The idea that the original 
five colleges would participate at no charge during the first year made sense, 
given that the entire project was experimental « However, the second year, 
those five colleges were each contributing $2000 and the next group of 
colleges was benefitting from what was learned the first year and receiving 
their initial year of membership at no charge. One of the colleges which 
joined during the second year dropped out after their free year. Colleges 
were required to sign an agreement to participate for one year as financially 
contributing members, but the agreement was not enforced. There may have been 
some feelings of inequity around aspects of the financial arrangements. 



Another problem we had not anticipated was the difficulty of working 
simultaneously with several sets of institutions (first year,^ second year, 
third year) each at different stages of understanding regarding both 
assessment and the consortium. The director's attention was d:^vided by 
having to explain and essentially market the project to potential new members 
while simultaneously developing and implementing initial plans and collecting 
and analysing data. 

Additional advance planning would also have addressed the criteria to be 
used for adding new colleges. We had originally thought that we could absorb 
ten new colleges each year* After the first year we had revised that to five, 
but when eight wanted to join we had no basis for selection, so all were 
added. In retrospect, more time and attention should have been devoted to 
planning before attempting to carry out the project. Both the deans of the 
colleges and some faculty and the University of Kentucky directors should have 
met over a period of several months to develop more agreeF««*nt about directions 
and more concrete plans. Because that had not happened, a great deal of 
effort and actual project time had to be devoted to developing a sense of 
shared direction and identity with the project. 

Background and origins of the project 

The Assessment Consortium is an outgrowth of the Appalachian College 
Program. Once a part of the University of Kentucky's Appalachian Center, that 
program is now called the Faculty Scholars Program and is administered through 
the Graduate School. The Appalachian Center itself was founded by John 
Stephenson, current President of Berea College, when he was a faculty member 
at the University of Kentucky, as a place where faculty scholars and 
researchers from various fields with interests in Appalachia could conduct 
interdisciplinary research which would also be of service to the region. The 
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Center included the Appalachian College Program which was originally funded by 
the Andrew W. Mellon Foundation. It provides fellowships for humanities 
faculty from private liberal arts colleges in the central Appalachian Region 
to spend time at the University of Kentucky studying and doing research which 
they could not do while teaching at their home campuses- Additional funding 
from the Pew Trust provides similar fellowships for science faculty. The 
interaction of scholars from Appalachia with those from UK enriches both the 
University and the region. The Program also funds grants for faculty to 
travel to professional conferences, workshops on computer literacy, and 
traveling UK scholars. Recently the Program has attracted funding for and 
implemented a regional computer network, an endowment for fellowships, and 
several science conferences. 

The Assessment Consortitun thus developed in the context of a history of 
cooperative working relationships between the University of Kentucky and the 
38 private liberal arts colleges in the five-state Central Appalachian 
region. However, the Appalachian College Program was essentially a loosely 
knit, voluntary group of colleges, whose deans attended an annual meeting at 
the University of Kentucky. The director of tb-* Program was interested in 
developing a project which would involve the colleges as institutions, rather 
than individual faculty from the colleges, in a mutually beneficial 
cooperative effort. Until the inception of this project, the colleges* 
cooperative effort had been limited to planning a conference on international 
education, which involved little or no financial commitment from the 
colleges. One goal for the project was to develop a consortitms which would 
involve more direction and commitment of both time and money from its members 
for their mutual benefit. 

Another factor in the background of the project was that the initial five 
colleges had been chosen by the director of the Appalachian College (Faculty 




Scholars) Program* mainly on the basis of her working relationship with the 
college deans. However, between the time when the colleges were selected and 
the project began, four of the colleges had new deans, so they were new to the 
idea of the consortium and to their colleges. The project was planned to be 
administered through the TJniversity of Kentucky. The project director was a 
higher education faculty member interested in types of higher education 
institutions and in using Entwistle's methods to examine student learning; the 
director of educational assessment was particularly interested in liberal arts 
education and had been director of institutional research at several 
institutions, including Berea College. They had originally proposed a project 
to assess learning in the major, because they saw it as a necessary component 
of an assessment plan and a relatively manageable context for the success of 
an assessment consortium. 

Project Description and Results 

It was not obvious at the time, but the college deans and the project 
directors started the project with unspoken but clearly competing visions of 
"a successful consortium." The private liberal arts colleges tend to be 
sensitive to being told what to do by the big university down the road which 
really doesn't understand them, just as the research university is resistant 
to being told what to do by liberal arts colleges which don't understand 
the^i Unfortunately, these divergent visions were not communicated at the 
outset. 

The deans saw the consortium as an action project. As the project began 
the five college deans were interested in assessing the outcomes of their 
general education programs and of the entire liberal arts experience, rather 
than their majors. General education is in some ways the specialty of these 
private liberal arts colleges. It would be helpful to be able to document 
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their successes in this area for their various publics. Assessing general 
education is probably the most difficult academic assessment; the deans felt 
that if they were going to have an extra boost from the FIFSE project, they 
should use it to approach a difficult problem. The fact that three of the 
original five colleges were a "cluster'* in the Association of American 
Colleges (AAC) FIFSE project on assessing academic majors further turned the 
group toward assessing general education. 

The project had two directors. The faculty project director had an 
extensive background in quantitative research in higher education and was 
primarily interested in generating research data relating to student learning 
in the Appalachian colleges, and he felt that it would be easier to develop 
comparable data by focussing on assessment in the major. He saw the 
consortium as primarily a research project. While the new deans probably did 
not fully appreciate the strength of the research possibilities, they were 
reluctant to devote resources to what seemed abstract rather than practical 
efforts. 

The other director also had an academic interest in higher education, but 
came from a backgrotmd in institutional research and educational 
administration « Having worked at Berea College and consulted with several 
similar colleges, she saw the consortium as an organizational problem of 
bringing together the academic research expertise and philosophy of the 
University of Kentucky with the academic teaching expertise and philosophies 
of the liberal arts colleges in a way that would benefit both. During the 
first year or so of the project there were some tensions related to differing 
expectations and assumptions. An advisory board met in Johnson City was 
scheduled and resulted in better mutual understanding and a much clearer sense 
of direction. 

Once it was agreed to work on assessing general education, it immediately 
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became clear that the deans had three major concerns* One^ they were 
interested in assessing the effectiveness of their programs, rather than the 
progress of individual students. Two, they were concerned about meeting the 
new SACS criteria, but they did not want to feel they were being forced into 
adopting a standardized testing approach to assessing general education to do 
so. And three, they wanted both the process and the results to be meaningful 
to the faculty, and to reflect the richness and personal quality of their 
students* general education experiences* They were intrigued by the idea and 
by the possibilities of serving as a reference group for each other, but a 
little wary about others knowing too much about their students. They also 
wanted to be able to compare their colleges with others nationally. . 

The project really began in November. The first few months of the project 
were devoted to collecting information from the colleges about their 
philosophy and goals of general education, and about the curricula developed 
in support of those goals. Course syllabi and final exams for courses in the 
general education core were also collected* This information was used to 
compare components of general education across schools and to think about the 
kinds of outcomes of most interest to the colleges* Assessing General 
Education had been the topic of the annual Shaker town Conversations on General 
Education (originally a sub-group of the Society for Values in Higher 
Education) that year, and outstanding educators from colleges and universities 
across the country had been assigned, in preparation for the meeting, to write 
some questions one could ask of seniors to assess their general education. It 
seemed that those questions addressed several of the deans* concerns; the 
source of the questions was a broadly representative national group, yet the 
questions were directly related to the goals of the Appalachian colleges* 

Helping students develop the ability to speak was a common goal of the 
colleges* general education programs, as was developing skills and knowledge 
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needed to be effective citizens in their conmiinity. Because students' writing 
is assessed many times during their careers ^ the idea of using those questions 
in interviews was appealing* It seemed an opportunity to assess students* 
knowledge and their skills in speaking, conducting civil conversations, 
following a line of thought and developing an argument or position in a way 
that would approximate situations of real life as college graduates. Further, 
it seemed appropriate to the personal quality of education students experience 
in small liberal arts colleges. The interviews were also consistent with the 
fact that most of the students are from Appalachia, a traditionally oral 
cul ture . 

During the first few months, then, the director visited each college for 
several days and met with the dean and the curriculum committee or the faculty 
council, sometimes with students, the registrar and director of admissions, or 
other appropriate groups* Interestingly, most of the deans wanted the 
Consortiuum director to introduce the notion of assessment on campus, and had 
provided little context to faculty ahead of time. 

During these visits it was necessary to explain what ''assessment" means, 
what the Institutional Effectiveness criteria might mean, and what the 
consortium might become. It was also important to discuss general education 
as it is carried out at that particular college, and how it might be 
assessed. Only at that point could one discuss the possibility of using 
interviews with seniors as one way to assess general education. Fortunately, 
the idea seemed to suit almost everyone. In fact, we discovered quite by 
accident that the idea of the faculty who had developed the curriculum talking 
with their own students had such strong face validity that it created a halo 
effect for other assessment data and activities. 

During these Initial visits we also conducted a sort of traveling Delphi 
technique with the questions. The director had culled and combined the 
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original set of questions, eliminating those that would obviously not work In 
an interview setting. The remaining list of questions was presented to the 
faculty and discussed at length* Faculty were very enthusiastic about some 
questions t unsure about some^ and clearly rejected others; after visiting all 
five, collgeSt a list of questions acceptable to all five colleges emerged^ 
along with a way to accommodate the variations within and between colleges. 

Naturally faculty had questions about how much time might be required to 
talk to a reasonable number of seniors* We decided that interviewing small 
groups of students had advantages of efficiency and student comfort » and 
agreed that on each campus about fifteen faculty would each conduct one 
interview with a group of three students. On each campus « A5 randomly 
selected seniors would be interviewed- Either as part of this visit or on a 
subsequent visit the director met with the faculty interviewers for an hour or 
two to train them and ensure the interviews were conducted according to the 
same general guidelines* (For an abbreviated version of the guidelines, please 
see Appendix A*) All the preliminary development and the actual interviewing 
itself took place within a highly compressed time frame* The interviews were 
taped and sent to the Consortium office for transcription and analysis* 

Transcribing interviews is a labor-intensive process* Each interview 
lasts about two hours* The first year there were 45 interviews (15 from each 
of five colleges) or about 90 hours of conversation to transcribe* On average , 
it appears to take A hours to transcribe one hour of tape* One major cost of 
the interviews was the transcribing time, about 360 hours at about $8.00 an 
hour* Professional transcribers earn about $11.00 and hour« but it is often 
possible to find a graduate student ^ staff person or even a work-study student 
who can transcribe for significantly less* It is helpful to have a person who 
is familiar with the names* customs and language likely to be used by college 
students* While the approximately $3000 for transcribing seems expensive, it 
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costs each college less than $600 a year^ a cost well below that for 
standardized tests such as the Academic Profile or ACT COMP. Obviously, if 
only one person did all the transcribing, it would have required nearly ten 
weeks of full-time work. 

Interviews are conducted during the spring of the senior year. The plan 
was to complete the analysis of the transcripts and have information available 
to feedback to faculty at the beginning of the fall semester. It became 
obvious that completing the interviews before spring break is useful from many 
points of view. We were able to find several very good graduate students 
during the first consortium year* In the second year, the consortium had 13 
members, and the work of transcribing was arranged for locally by each 
college. 

Transcribing is only part of the work. The reason for transcribing the 
interviews, of course, is to allow for systematic analysis so that others can 
share the information and use it as part of the basis for assessment and 
planning for change. Analysis of the interviews was done by the consortium 
director so that comparisons could be made between colleges. 

The method of analysis was to take a pheiiomenological/ethnographic 
approach. These interviews are intended to give students a chance to 
demonstrate their general education, rather than to discuss it, so in a sense 
they arc a kind of performance assessment. It is important that the analysis 
be carried out by someone or ones fairly knowledgeable and familiar with the 
college. The reader generally takes all the transcripts from a particular 
college and reads the entire set, much as if one were reading a novel. 
Sometimes this process is repeated several times, so that the flavor of the 
whole group comes through. Then the goals for the general education program 
of the college are used as a basis for analysis. The comments which relate to 
a particular goal are compiled. They are then arranged on the word processor. 
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into groups of those which indicate that a goal is being achieved regularly 
with a reasonable level of quality, those which indicate otherwise » and those 
which are conspicuous by their absence. Each general education goal is 
treated the same way. (Obviously, the interview is not the best technique for 
assessing students ability to use certain mathematical skills and 
principles. ) 

Another part of the analysis is a summary of responses to each question, 
with an explication of the patterns of content and thought processes most 
often demonstrated. In the process of this analysis, other kinds of themes 
and issues frequently emerge. Often they are not directly related to the 
questions asked by faculty, nor to the goals of the general education 
program. Yet they are often important enough to note and report for the 
benefit of the deans and faculty. 

The interview transcripts are rich sources of data, and can be analyzed 
in many different ways. For example, they can reveal patterns of 
gender-related attitudes and interactions or attitudes about various campus 
sub-groups such as foreign students, racial or religious minorities, or 
fraternities. They may show differing values or definitions of an "educated 
person" by majors. Given the limited time available for the director to do 
all the analysis, this type of analysis was not undertaken. Rather, when the 
analysis for each of the colleges is completed, it is possible to make some 
comparisons between colleges. Although the major comparison is between the 
goals of the college and the responses of its students, the goals are often 
similar enough that comparison across colleges enriches the information and 
makes it even more useful. 

A written report is prepared for each college and sent to the dean. It 
is important that the analysis also be provided to all the interviewers. 
Usually the consortium director would meet with the appropriate group from 



each college — the interviewers » the general education coiiMnittee» or the 
entire faculty — early in the fall to go over the analysis and involve them 
in expanding the analysis. Since the curriculum and the process are both 
owned by the faculty, they are the real authority, and it is important for 
them to know that while the analysis gains objectivity by being done by an 
outsider, certain comments and events require their perspective for accurate 
interpretation. The director would repeat this process for each college, 
while simultaneously collecting suggestions for improving the interview 
process and questions in the next cycle* 

The first year all the transcribing was arranged centrally through the 
consortium office. The next year, when there were thirteen members, the 
colleges managed the transcribing and sent the transcripts — both on paper 
and on computer disk — to the consortium office for analysis. One college 
decided to videotape their interviews, and rather than transcribing them, have 
developed a rating sheet which is used by a group of faculty who view the 
tapes for analysis* A faculty member from another consort iimi college is 
included among the raters to help check their interpretations and conclusions. 

The project dissemination conference will incl\jule sessions on analyzing 
the interviews so the process can be taken over by the colleges themselves. 
Another assessment approach used by the consortium was the Academic Profile. 
Initially the deans opposed the idea of measuring general education with a 
standardized test because they believed that a standardized test would not 
capture the richness and depth of their general education programs and because 
they felt that national norms sometimes fail to reflect the real 
accomplishments of their students and faculty. However, the possibility of 
participating in the pilot testing of the instrument and serving as a 
reference group for each other seemed to be worth trying. The first year the 
colleges collectively learned a lot about the problems of motivation and 
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administration of standardized assessments. Unfortunately, the information 
generated by the tests was difficult to use. The composition of the norming 
group, in terms of institutions and atudent level was not available, and each 
college simply received five numbers. The fact that the consortium knew which 
students had been tested made that information a little more useful because 
some general comparisons could be made and related to the average ACT scores 
of the tested group at each college, although at some colleges the composition 
of the groups was not recorded. There was a high correlation between entering 
ACT scores of freshmen and Academic Profile Scores of sophomores. By the end 
of the first year we had learned something about how to present the test to 
students and deal with problems of motivation which could be shared with each 
other and with colleges which joined the consortium during the second year, 
but otherwise the cost of time and energy exceeded the benefit of the 
information received from the Academic Profile during the first year; in its 
second year ETS made some adjustments In the test and ir. the Profile. The 
test was shortened to fit into a standard course time slot, and the results 
were in a more usable form. By the end of the third year, there was still not 
enough Information to warrant changes in general education programs, but it 
did provide some general indicators about areas that might need to be 
examined. Several of the colleges are continuing to use it with the hope that 
the information which allows them to compare the same group of students over 
time will be more beneficial to the individual colleges and to the 
consortium. 

The project turned out to be more complex than originally anticipated, 
for several reasons. For one thing, each issue had to be addressed at several 
levels. For instance, some decisions about assessment were made by the deans 
and project directors as a group. It was important to educate the deans about 
assessment early in the project. This of course involved Intensive 



conversations about the nature of assessment and the kinds of questions the 
deans considered important These conversations were equally Important — in 
fact, essential — in building the trust necessary for cooperative efforts In 
order for the project to succeed, the deans had to trust both the director and 
each other in regard to questions of data confidentiality, balancing the 
interests of the whole group with those of individxial institutions and of 
individual faculty within the institutions. The deans had to be willing to 
share their weaknesses as well as their strengths* Their knowledge, 
experience and ideas are absolutely essential in providing direction to such a 
project, and to the ultimate success of any cooperative effort. 

In such a situation the directors can only direct if they can first 
inspire the participants. This had to occur with both the deans and the 
faculty* The deans were aware of the potential value of assessment to their 
colleges, but their levels of comnnitment to the consortium varied, as did 
their ways of communicating and involving faculty within their colleges. At 
the same time, a plan for carrying out assessment as a consortium had to be 
developed and implemented. While that was happening, the director was also 
involved in recruiting new colleges for the flubsequent years. 

The deans and assessment coordinators made up the project advisory board, 
and most of the deans had at least a professional acqtiaintance through the 
Appalachian College Program's annual Deans Meeting. Even so» most of their 
meetings were in a context which required very little effort on their part. 
With the consortium they had to engage in discussions of policy and 
procedures, and a new kind of trust developed. After the first eighteen 
months of involvement with the project, several deans expressed surprise at 
the power of cooperation, as opposed to their typically more competitive 
relationships. 

While there were existing occasions when the deans could meet together. 
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those occasions had to be created for the faculty. The consortium Assessment 
Institute and Assessment Conferences served that purpose. During the first 
project year 9 a two-week Institute on Assessment was conducted at the 
University of Kentucky. The Institute was seen as an opportunity for the 
original colleges and others which anticipated joining the consortium to send 
one or two faculty to Lexington to discuss in depth some of the theoretical 
and practical issues in assessing the educational outcomes of college 
programs • 

The primary faculty for the Institute were Professors Leonard Baird and 
Edward Kifer, both of the Department of Educational Policy Studies and 
Evaluation* Baird *6 primary emphasis was on the relevant research literature 
on higher education and assessment, while Kifer focused more on issues of 
using existing data» and questions of exploratory data analysis using the 
personal computer. Other resource people including Jonathan Warren, Andrew 
Grimes, Beth Goldstein, Jim Rodgers, Karen Carey and Charles Elton, addressed 
course-based assessment, ethnographic approaches to assessment, issues in 
assessing organizational effectiveness, inter-institutional comparisons using 
quantitative data, and the Southern Association's expectations regarding the 
Institutional Effectiveness Criteria. The Institute was held in June of the 
first project year. Only three of the fifteen participants were faculty from 
colleges which had oeen in the consortium the first year. The others were 
from other colleges in the Appalachian College Program, colleges which joined 
the consortium during its second year. 

During the second two years the consortium arranged two two- or three-day 
conferences each year for particular groups of faculty. The usual attendance 
was about forty faculty and three or four deans. The first on^ was a 
conference on classroom level assessment with Bud Warren held at Marsrville 
College. It was designed to respond to comments of those who attended the 
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Assessment Institute^ who wanted more time to pursue Warren's ideas. 
Participants were faculty who teach the introductory course in the humanities^ 
biologyt and psychology which is part of the general education proerara^ 
Faculty brought copies of syllabi ^ spent some time taking about asking good 
exam questions « and worked together describing their common course and general 
education goals and developing questions which would really help them 
ascertain the extent to which they were being accomplished » and working with 
Bud Warren to see ways to make comparisons across disciplines. 

A second conference that year evolved from a deans* discussion of their 
uncertainty about how well their colleges were doing in terms of the aesthetic 
outcomes of their general education curricula and their consequent wish to 
involve arts faculty in assessment. We had planned to rotate the locations 
through the members' campuses, but that conference was held at Pipestem State 
Park in West Virginia, which turned out to be an ideal location. The 
conference was opened with a talk by Professor Phil Alperson from the 
University of Louisville, whose combined experiences as a professional jazz 
musician, member of the Board of several local and statewide arts 
organizations, and philosophy professor specializing in aesthetics became a 
touchstone for conference conversations. The main conference work was done by 
panels of faculty from consortium colleges, however. Arts faculty were very 
enthusiastic about trying out some of the assessment ideas they had discussed 
and getting back together to compare results. In addition, several faculty 
volunteered to develop and circulate lists of arts resources and events which 
might be made available to the other colleges, such as exhibits, faculty 
recitals, student theatrical productions, and other special events. 

To follow up with art faculty and extend the assessment dialogue, the 
third year the consortium sponsored a conference on assessing the artistic and 
scientific understandings resulting from the general education curriculum. 
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Dr.* Patricia Kerr from the Center for Teaching and Learning at the University 
of North Dakota helped focus the conversations by introducing concept mapping 
and knowledge Vee diagramming as a way to help both faculty and students 
penetrate the structures and meanings they seek to xinderstand^ and to help 
them articulate the real outcomes they wish to achieve and assess. Both arts 
and science faculty were involved in the kind of serious conversation which 
happens too rarely » according to their evaluations. Several participants 
commented that they also had gained a new appreciation of colleagues from 
their own campus partly by virtue of talking and traveling together* and 
partly by seeing them in a new context. Faculty gained a renewed appreciation 
of their own college's strengths in addition to gaining a new approach to 
assessment* 

That conference wap followed by one planned to address assessing the 
science and religion goals and outcomes of the general education curriculum. 
Part of the conversation involved the science faculty and religion faculty 
developing general education goals for each other and articulating why it 
mattered to them what their graduates understood about science or religion* 
Both groups were somewhat surprised to find that it was almost equally 
difficult to identify the kinds of science or religious understandings they 
hoped for their students. Their cot^ rsations became the basis for two 
conferences held after the FIPSE funding ended. The importance of the 
relationship between knowing personally and knowing about something, i.e., 
religion, academically, led to plans for a conference on assessing student 
development outcomes as applied general education. Frustration with the fact 
that roost graduates of these colleges are not science majors yet they will 
need certain kinds of scientific understanding to live in a highly 
technological and scientific environment, led to a conference on developing an 
integrative science course for non-science majors. 
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The conferences accomplished several goals* One was to infonn« to 
communicate the major ideas of assessment and to engage faculty in 
conversation about these ideas and their implications frr teaching and 
learning in the context of genrral education. Another was to empower 
faculty. At several colleges the faculty are highly tenured and isolated to 
the extent that they have lost perspective on their own skills and 
understanding. The conferences were partly intended to give faculty a renewed 
sense of their own ability to approach education creatively and of having some 
genuine understandings to offer faculty from other colleges as well. 

The conferences were also a means of spreading assessment as a way of 
thinking further into the faculties « as each conference involved different 
groups of people. At the same time» some overlap in attendance from one 
conference to the next wes planned, so that ideas from one could be addressed 
at the next, and faculty could follow up with each other from the previous 
conference. From the coordinator's perspective, the conferences also helped 
faculty develop a feeling of being part of a consortium. When people actually 
do work together they develop a sense of ownership and participation that is 
not possible when the deans simply make an agreement and the director visits 
the individtial campuses. Faculty also had the opportunity to exchange ideas 
with colleagues and to compare programs in a way that helped them assess their 
own strengths and weaknesses. Conferences also provide time when faculty can 
discuss ideas and modify plans to incorporate what others have learned in 
trying similar approaches. Although outside speakers were incorporated into 
some of the conferences, the goal was always to empower faculty through their 
understanding of their own academic authority and expertise. 

In many ways the conferences became an effective and relatively 
inexpensive form of faculty development. An interesting unintended outcome of 
the conferences, according to several deans* was that sometimes people who 



were known on their own campus as uncooperative, disinterestedt and irasciblet 
were creative and vital participants in the conferences* Faculty expanded 
their network of colleagues within the region t and other activities were 
generated through the conferences as well. As a result of the Institute and 
the Naryville Conference » one faculty member became very interested in Bud 
Warren's course-based assessment and was invited to participate in a project 
with three very different types of institutions across the country and present 
the results at a national forum* Psychology faculty began work on developing 
an undergraduate research conference for their majors* and science faculty had 
a special conference on developing an integrative science course for 
non-majors. 

The consortium also adopted another assessment instrument. The College 
Student Experiences Quest ionnaire* developed by Robert Face of UCLA, is used 
to provide context for interpretation of the interviews, as well as to give 
the colleges a sense of whether their students behaviors as they report them 
match the assumptions the colleges make* One section of the questionnaire 
asks the students to report how much they have gained in various areas, while 
most of the questions try to get at the quality of students effort in various 
acaiemic and non-academic aspects of their college experiences. Even though 
the questions are standardized in format, they are self-reports of behavior* 
and in that sense the data are qualitative. However, the fact that there are 
national norms for each scale by types of institutions makes the information 
useful for comparisons within and outside the consortium. In most cases the 
Pace data confirm other information. For instance, one college had a high 
percentage of part-time faculty and substantial turnover* The number of 
conversations with faculty outside of class and the content of the 
conversations differed significantly from most other consortium colleges. 
Even though it is not asked, students ustially mention particular faculty by 




name in the senior interviews. In the interviews from that college no one was 
mentioned 9 so the multiple methods tended to confirm each other, as was 
frequently the case* The colleges have found the CSEQ data to be worth 
continuing, as it provides a link between the academic and student development 
outcomes of undergraduate education. 

The freshman-senior essays were not started until the second year of the 
project, and not all member colleges were involved. Those who were will have 
senior essays to compare with their freshman writing samples early in the Fall 
semester of 1990, when the information and the usefulness of the approach can 
be evaluated. 

Overall, what did we learn? Wi» learned about consortium building « We 
learned about assessment of general education at the program level. We 
learned about interviewing. We learned about gaining student and faculty 
cooperation and interest* We learned about the colleges, their students and 
their curricula. And yet, because of the nature of the consortium, it remains 
difficult to say who or what changed as a result of all this, and how* 
Probably the most important change will be in the way faculty think about how 
they teach and how students learn, and while it is very likely that those 
changes will be partially related to the activities of the project, it is 
unlikely that the relationship will be documented* 

A few general examples might convey the flavor of what was learned. One 
question asked in the senior interviews was 'Vhat have you learned about a 
non-western culture that helps you understand more clearly your place in the 
world?** Most of the colleges have as a stated goal of the general education 
program to increase students* global awareness and understanding of 
international perspectives, but very few of the students from any of the 
colleges could respond to that question in much depth. Students at one 
college responded almost entirely from the perspective of European culture* 
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The one college with a truly international flavor also has a required service 
project, which often occurs in a third-world country* The colleges are less 
than an hour away from each other, but the responses of their students were 
quite different. How did the colleges respond to this kind of information? 
One of the deans reported that the conversation in faculty meetings and 
curriculum committee meetings has changed significantly* Faculty are now more 
careful to question their own assumptions about what students are learning and 
how. They have begun referring to what they have learned from the interviews 
to support their positions* Another dean reported that« while they did not 
have the resources to significantly add to the faculty or change the 
curriculum, they wanted to keep global awareness as one of their general 
education goals* They did have some faculty development money, however, and 
were able to use it to send a group of faculty who teach general education 
courses to China for several weeks during the summer, so that all their 
courses could be informed by that perspective, and it would be reinforced for 
students through several disciplines. Another college responded to this 
information by initiating a major review of their general education program 
and goals, and by making a course in non-Western culture a mandatory rather 
than optional element in fulfilling general education requirements for a 
degree* Others have done nothing except feed the information back to the 
faculty councils for their discussion, with the assumption that faculty will 
individually make adjustments in their ovm courses. 

Another question used in the interviews was **What is an educated person? 
and "How close are you to being one?" There were some distinct patterns of 
responses to this question, which tended generally to reflect the emphases and 
values of the particular college, and to involve depth, breadth, social 
responsibility and a distinction between practical intelligence and 
education* It was possible to generalize across colleges, as well, to 



conclude that students have a rather diffuse set of notions but they do not 
have the conceptual vocabulary needed to respond as some might have hoped* 
Rather they tend to respond very personally and descriptively, with little or 
no reference to ideas or information from philosophy, education^ psychology, 
or history. It appears that education is not a topic of discussion, and that 
students generally lack awareness of their own education as part of an 
evolving historical and cultural process* Several colleges have made plans to 
address this by changing their freshman and senior seminars somewhat, and by 
asking faculty to be more explicit in bringing educational perspectives to 
discussions in various disciplines. 

One question asked students to respond to some element of the general 
education program. It is not uncommon for students to begin their response as 
though they were freshmen, and in the course of their discussion, make 
statements about connections between courses and ideas that they had never 
made explicit for themselves before. The fact that the interview process 
itself is also inherently educational is valued, at least in retrospect, by 
both students and faculty. 

Evaluation 

It is difficult to evaluate a systemic project which operates on many 
levels with multiple goals. Evaluation has been built into the projects many 
aspects from the very beginning, with the understanding that it would be 
primarily conversational, formative, and continuous. Every facet of each 
element of the assessment consortium has been discussed and refined in the 
light of the discussion. The interview questions, process, analysis, 
transcriptions have all been evaluated in this manner. The conferences were 
evaluated in writing, usually with an open-ended form. Copies and/or 
summaries were sent to the deans of each college, and the information was used 



to help plan subsequent conferences* 

When the project began one stated measure of success was that the 
colleges would continue to fund the consortium after FIFSE funding ended* The 
colleges decided to fund the consortium for an additional yeart with a review 
at the end of that time to determine whether the consortium will continue and 
if so, under what conditions* The consortium is now administratively located 
in the University of Kentucky's Faculty Scholars Program Office, under the 
direction of Dr* Alice Brown. The Consortium is now staffed by two 
professional people, each working half-time* The Director is Dr* David 
Knisley, on half-time leave from the history faculty at Mars Hill College. He 
served as Associate Dean for Evaluation and Instructional Design as Mars Hill 
was implementing its competency-based curriculum, and has consulted with other 
colleges on related issues. The coordinator is Stacy Street, who manages most 
of the conferences ^rd internal affairs of the consortium office. The fact 
that it is located at the University of Kentucky continues to provide 
stability and a commorx connection. 

One of the original goals was to develop a common computerized data 
base. While it was possible to collect and compile some data at the 
consortium office, the existing computing facilities at the colleges were so 
diverse that people did not seem to be interested in devoting the time and 
energy required to make such a data base useful. In the meantime, however, 
the Faculty Scholars Program has obtained funding through AT&T, the 
Appalachian Regional Commission, the Tennessee Valley Authority and the 
National Science Foundation for a computer network which has been installed at 
ten of the colleges with the assistance of several University of Kentucky math 
and computer science faculty. It has not yet been used in conjunction with 
cooperative assessment activities, but at least one group of faculty is 
already planning to do so. 
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The computer network also helps in achieving another of our goals* which 
was to help change the relationship of the University of Kentucky to the 
private liberal arts colleges in the region* The consortium has provided a 
possible model for members of the Appalachian College Program to be related to 
each other more actively and to strengthen private higher education in the 
Central Appalachian region. John Chandler of the AAC conducted a major 
evaluation of the program and concluded that the time may be right for the 
colleges to form a more independent organization* and the deans and presidents 
of the colleges are currently discussing and designing such an organizational 
structure. The project has made it possible for them to envision such an 
organization as a reality* and to anticipate some of its directions and 
activities . 

Dissemination 

Information about the project has been sent in response to direct 
inquiries generated primarily through the directory of Assessment Projects 
compiled by FIFSE and presentations made at national meetings* Several 
colleges and universities* including the University of Kentucky* Ball State 
University's history department* several community colleges and individual 
honors programs and student affairs programs have adopted the interview 
process and some of the questions as part of their assessment programs. 

Presentations at professional meetings have included the Kentucky 
Association for Institutional Research* the Kentucky Forum on Faculty 
Development* The AAHE Assessment Fortmi* the Annual Forum of the Association 
for Institutional Research* the North Carolina Association for Institutional 
Research* and the annual meetings of the Appalachian College Deans. Several 
articles are also in process. 

In addition* a major dissemination conference is planned for September* 



1990 in Lexington, Kentucky. The conference will involve faculty and deans 
from various colleges as well as several sessions on how to analyze interview 
transcripts. For more information on this conference! contact Stacy Street t 
Appalachian College Assessment Consortium » 110 Maxwel'4;on Court, University of 
Kentucky, 40506-0347. 

Smmary and Conclusioaa 

Developing the Assessment Consortium has been challenging and rewarding 
at every level. The consortium has involved fifteen colleges. About three 
hundred faculty participated in conference activities, and another two hundred 
served as interviewers. Fifteen college deans have worked together, along 
with about ten University of Kentucky faculty and several nationally-known 
consultants. Several informal networks have developed among faculty. 

The Assessment Consortium has developed an apparently unique qualitative 
approach to assessing general education programmatically . With minimal 
publicity it has generated hundreds of requests for information about both the 
interviews and issues of inter-institutional cooperation. Many students have 
echoed the opinion of the Berea student who wrote to the College newspaper 
about the value of the interviews. (His only complaint was that not all 
seniors were interviewed and that students at all levels should be includeu.) 
Indications are that this kind of cooperation and the use of qualitative 
evaluation will become increasingly important in the future. 

The Assessment Consortium has reached a point from which it can become a 
unique and valuable organization in its own right, as well as a model for 
others in similar kinds of colleges. Whether it does so will depend on the 
colleges' willingness to take some risks, to make a stronger financial 
commitment as well as a further involvement of time and energy on the part of 
the deans. The program needs stable financial resources to support the 
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leadership necessary to continue and expand asseasment activities and for new 
ideas to take root and to grow organically over time* 

Our major advice for people considering such a project would be that both 
the process and the results are well worth while. Cooperative efforts are 
just that — they must be cooperative and they require a lot of effort. But 
with a good plan and communication system^ they are worth the effort. There 
are very few models or experts for guidance » so it is essential to talk to 
other people and to each other. When the entire situation is new it is 
important to balance the needs and wishes of the college deans with the need 
for central coordination. 

Devoting time to careful conversation is essential to building the kind 
of trust and understanding needed Planning at the outset can simplify the 
central coordination tasks » and central coordination, including leadership, 
perspiration and, perhaps, inspiration, is crucial. Although the director has 
the final responsibility for the project and the authority of certain kinds of 
expertise, the real authority resides in the colleges. By effectively 
combining the responsibility and authority into a functional organization, a 
consortium has the power to make a significant difference for private higher 
education within a region, and for the individual colleges, faculty members 
and students as well. 
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Appalachian College Assessment Program 



Assess i ng Genera 1 Educat ion 
Interview questions for seniors 

Purpose 

1. To help the colleges learn i^ihether their graduates 
exhibit a constellation of skills, knowledge and attitudes whict) 
reasonably represent attainment of the college's stated Qoals a»Kl 
which might be attributable to the general education proara^M of 
this particular college; 

2. To help faculty in personally and directly assessing the? 
effectiveness of the general education curr iculum/ 1 iberal arts 
experience of the college's graduating seniors; 

3. To help the colleges to identify areas of strenqtf^ or 
«^ieakness as a basis for planning and tc provide perspective tor 
faculty discussions and decisions regarding the general edticatiori 
curr icu lum * 

^. To help the consortium see whether the seniors ^^»ow 
certain qualities or values as a group which might be 
attributable to this particular type of college^ and I'ihether 
differences between colleges are related to differences lo the 
content of the general education program. 

Preparation 

1. Format: Approximately fifteen faculty members at each 
college will conduct a conversation/ inter view/focus group. Each 
group will consist of a faculty member and three seniors. None 
of the seniors will be majors in the faculty person's discipline. 
Ideally the seniors will be from different majors as well- 

The reasons for this arrangement are: 

a. the time required from each faculty member is 
mi nimal 

b. students are more natural and comfortable in a scnall 
group than in a one-to-one taped interview with a 
faculty member and they are stimulated by each 
other ' s ideas 

c* the mix of majors makes it easier to avoid a 
discipline-focused conversation ; 

2. Faculty: Interviewers should represent a cross-section of 
majors* Other selection criteria might include: involvement in 
general educat ion? skill in facilitating group communication and 
sensitivity to students. 

3. Students: Seniors should be selected randomlv. Select 
more students than you actually plan to interview* as there 'Mill 
invariably be some who can't participate. Any volunteei 
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selection process will systematically Dias the results and ti«e 
value of the conversations for the college. Becausr^ it coincidrrs 
with their strong feelings about reaching the end of their 
undergraduate education and their need for closure and synthesis, 
it is possible that a few students who are not in the randOfMl. 
selected sample may feel slighted. Publicizing the procet.'^ cihcad 
of time on campus can help prevent this and positively influei^ce 
students toward the entire assessment effort. 

^. Matching faculty and students: Some colleger o< e 
interested in looking at gender or ma jor^r el a ted influence^ or. 
the conversations- If this is the case^ a plan for con'3 1 i tu 1 1 « iC 
the groups should be developed after the random sarnpie 
selected . 

5. Par t ic ipAt ion: Students to be interviewed should be 
notified in writing that they have been selected^ how ar^d . 
Thev should know how much time to allow* where and i-meri to 
appear * and whom to contact if they have questions or proble^oe 
about participating. The letter should stress the importaoco of 
the project for the college and the fact that, due to the 
selection process* each student represents several other 
classmates as well as himself. <See sample letter enclosed.) 

6. Questions: For purposes of the consortium, use any two of 
the first three questions on the attached sheet. The deanf 
assessment coordinator f or facul ty commi t tee may have devel oped 
other questions specific to your college* Some colleges <na v 
want all interviewers to use the same questions* while others niav 
leave the choice to the individual faculty member. 

7. Physical arrangements: The interviews should be conducted in a 
room with no telephones or other extraneous noise or distraction* 
The best conditions for taping would be four people sitting 
around a small round table. Open windows near lawn mower s« 
construction work* fire sirens or railroad tracks can create 
major problems. 

Pleasv test the recorders before using them. Record at oiediuni 
to high volume. If possible* plug recorder into an electi icai 
outlet rather than using batteries. If you use batteries* pJea^se 
supply an extra set to interviewers with the recorder. If yoo 
use a micro-recorder f be sure to record on normal speed. 

Use good quality tapes. Maxell and TDK seem to work well. 
Last yebr transcribers had difficulty working with some of the 
really inexpensive variety^ and some of the data were lost 
because of this. Try to keep the interview to approximately \ i^o 
hours. This suggests that 120 minute tapes would be ideal - Li"*" 
transcribing requires a lot of stopping and reversing the tape* 
and ISO minute tapes are so thin that they tend to stretch ^^id 
tangle. Either 90 minute or 60 niinute tapes aie fine. Yo«jt 
tapes will be returned to you. 

Faculty interviewers should arrive early and be 
everything is in order. 

BEST COPY AVAILABLE 



Conducting th« Int«rviei* 

1. Before che taping begins* introduce yourself to the 
students and briefly explain the project. Stress the importance 
of their involved participation and try to oiake them coinfortabU 
with the idea that their honest conversation will be o-f the 
greatest value to the college. Assure them that nothing thev ^=ay 
can affect their standing in the college in any way, and ask t\\e^n 
to treat their classmates' comments with respect. Explain tNat 
your main role is to facilitate conversation among them and tf.at- 
while you might interject a comment or question, they are to talk 
with and question each other. You want to avoid a series of 
separate conversations between you and each of the students. 

a. To help test the recording level and speaking level, ctsk 
each student to sav a few sentences, including their first nahie:. 
major, hometown, and what they expect to be doxng this time next 
•/ear. Play it back and make any needed adjustments. 
This will be a great help to the transcriber, who will l-.ave to lie 
able to distinguish the three voices without knowing the peupifr. 

3. Give each student an index card. Tell them what the 
first question will be. and ask them to take a fm*t minutes to 
write a response. Turn on the tape recorder, pose the first 
question, and ask each student to read his response. The 
advantage of this is that it makes it impossible for them to 
avoid conversation by saying "I agree with her." 

i¥. Under each of the questions are included some possible 
follow-along questions. You may want to use one, none or all of 
them, or you may think of better ones. They are provided as a 
way to get students to go further with a question without 
prompting them to give a particular "right" response. 

5. Use at least two of the first three questions, with earl« 
student having the opportunity to be first respondent. Invite a 
student by name to respond first to each question. 

6. Do not interrupt students while they are talking. If 
they interrupt each other, allow it unless someone constantly 
interrupts and another person never gets to finish a thought. at 
an appropriate place in the conversation, go back to a point if 
you think someone wanted to add something but lost the 
opportunity. Specifically ask that person if she wants to 
elaborate on a point or add something to it. 



7. Keep track of time. The 
should be two hours, but that is 
deeply involved in a discussion. 



maximum length of the intervie*'* 
not required. If student'3 are- 
let the discussion take a 
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n«tur«l cour««. Oth«rMis«t aIIom far aoout 20 minut«« for c«cn 
of tn« first thrctt topics. 

8. At th« end of th« intvrvivws 

«. Thank students for th»lr h«lp *nd *nsw«r *ny 
questions. 

b. P1«M» be sure taoee are labelled appropriately t 
Mith the College namet datet (Spring 60 )t 

interviewer's name* and first names of the students. 

c. Complete the evaluation sheets* ^Se^ 

d. Return sheets and tapes to campus coordinator*. 

9. Before the end of the eemestert all faculty mHo conducted 
intervieMS should get together to peel thel" Misdo« in so«e 
syetemetic way. If it is written up* please send a copy to Karen 
Carey t 111 Oickey Hal It University of Kentucky* Le»lngten« KY 

40S0S. 
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Aoo*l*chi*n CoUeg* Assessment Program 
Spring 1998 Interview Oucmtions 



I. What do you consider an educated person to be? 
Possible follow-along questions: 

a. How close are you to being one? 

b. Are there certain qualities essential to being an 
educated person? Can you relate thes» to the ideas 
people you encountered in your general education ca 

c. If an educated person must possess certain kinds of 
knowledge or information, what might that include? 
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a. Sometimes you hear a particular person * 
"Citlxen of the world.* Thinking of yourself, what have you 
learned here that would help you understand your place In the 
Morld and how you might be an effective citizen In It? 

Possible follow-along quest lonsi 

a. Was a particular course or faculty 
member important in this regard? How? 

b. was your understanding influenced by any college 
ewperiences outside of class? 

c. HOW, if at all. will you will you live differently 
because of this understanding than you might have lived 
otherwise? 

3. What do you see as the role and/or value of the artist in a 
society or culture? 

Possible follow-along questions: 

a. Why are artists sometlm#!S perceived as being on the 
fringe of society? Can you give an example? 

b. wny are works of art often the first things to be saved 
in the r-'ent of fire or war? 

c. Is the value of art in what it does for the artist in the 
process of creation, what it does ^or^^* V'V'rZV .IZ.mtZ 

BESrrnPYAVAIIARIi: Prozess of experiencing it, ^nat it doea for the society 

^BEST COPY AVAILABLE culture over time? 

d. HOW has your college experience with the arts influenced 



yOMP r«spons« to this question? 



9. Does 3UP Ability to reproduce art mecnanical ly i i.e.. Ov 
printing, by recording, by photography, change its v*lue? 
HoM or Mhy? 

u. What is work? Uhat is the meaning of work for you? 
Possible fo I lOM-along questions! 

A. If there a difference between your Mork« your job. and 
your career? 

b. How are your values and your work related? 

c. Do you choose your work or does your work choose you? 

d. Does the element of choice have any bearing on what work 

is? ^ ^ r» 

e. How would you have answered this question as a freshman? 



3. What does freedom mean, and how is it enempllfied in 

individuals and in society? 

Possible follow-along questions: 

a. When you think of a person who is freei or not frest what 
do you see? 

b. When you think of a society that is frest or not freet 
what 00 you see? 

c. Are there certain conditions that allow or inhibit 

freedom? 

d. What from the general education core informs your vision 
freedom? 

6 What were two or three of your most important college 
OKperiences? (These expsriences might be positive or negative, 
academic, social, spiritual, personal, cultural.) 

Possible follow-along questions! 

a. How were you changed by that? 

b. What insights or knowledge could you share with a 
freshman at this college that no one else might tell him 
or her' 

7. YOU have jjst won a million ta^-free dollars. How do you Dl»n 
to spend it and wr»y? 

Possible follow-along questions: 

a. What would you base your decisions on? 

BEST COPY AVAILABLE 
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t). Ap« th«r» any oth«r waym to sp«nd it that would be 
fflttAnlngful to you? 

a. wn«t kind of Morld do you think your gr«ndch i Idrwn Mill liv» 
in? That Mould b»« p«rh«p« 33 to **0 y««r« from now* in about 
aoaa. 6lv«n that, what character i«t ic« would you hopo to in 
your grandcht Idron? 

Posmibls follOM-along quastions: 

a. Can you r»lat« this to conoptions of stability and 
changv as you understand thsm from your gsnvral vducation 
coursss? 

b. Uhat aro ths impiications of your rsspons* for your life 
or for porsonal action? 

c. If you pian not to hav» childron* Mhat doss this decision 
havs to do with your vision of ths futurs? Wlhat doss it 
havs to do with your vision of yourself? 
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MARYVILLE COLLEGE 



Each of thm following is on« of thm goals of tr\m college. 
On the basis of your interview alon«« p1«as» rat# on a scale 
of l-S how well the college is succeeding at achieving eacn . 
of these goal s • If you cannot make an evalua t ion« mark an 
•'X" on that line. Use ths space between items to record 
e^amp les • 

Scale 

l«very a«fairly 3-adequat«ly ^-poorly 3«not at X«can't 
well well tell 

ATTITUDES 

1- A sense of wonder and willingness to explore that 

will facilitate life*lonq learning. 



2* attitude toward leisure which recognizes th« 
necessity of recreation, the value of creative playt and the 
opportunities for personal growth and social servlc«# 



-^.•^•3- ^ well-developed social conscience and a commitment 
to responsible citizenship. 



Self-confidence, poiset and courage in the face of 

complexity, change, amOigui ty and adversi ty. 



3. Sensitivity and responsiveness to the individuality 

and needs of persons of other cultures as well as one's own. 



An outlook on one's life which Drmgs together its 

rational , emot ional, physical and spir i tual aspects « 



7. A view of one's self as Deing in relationship i^itn 

nature, society and Civimty. 



An attitude tov^ard work Mhich embodies a sense of 
vocation, permits tne satisfaction of accomp l i shment • and 
recognizes the necessity and dignity of laoor as a means of 
service to present anp future generation. 
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SKILLS 



1. Ability to oDs«rv*f r»#a and listen with disc«rnm#nt. 



Ability to think critically and analytically on 
matters both concrete and abstract. 



3. Ability to see r e 1 a 1 1 onsh i ps » to theorize, to 

synthesize information. 



Ability to express oneself clearly and persuasively 

in writing and speaking. 



^3. Skills necessary to interpret and use quantitative 

i nf orma t ion * 



6. Sufficient sw i I I ma second language to carry on 

basic communication. 



7. Ability to accomplish specific tasws and larger goals 

with little or no S'-»serv ision; to oe self-reliant* 



8. Ability to v^ei;n matter* t^cral c-^cice and ar r i v# 

at respons ib 1 e cone 1 : ons . 



Skills to 3ea*"in »or ana obtain employment suitable 

to one's talents a^c interests. 
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KNOlJLEOGE 



Understanding of tn» princip*! aopro«ch«« to 
knowlcdg* in ord«r to «ppr«ci*t« th» compl«« r» 1 At i on«h ip» of 
«11 knowledge «nd comprehend the tenta t i vene«s of all 
knowledge . 



« — 1-2' ^" understanding of man am an aesthetic creature. 



3- Knowledge of western history adequate to promote 

historical thinking and perspective. 



Sufficient acquaintance with a society or culture 

outside one's own tradition to define oneself more fullyi 
appreciate the other culture and cultural diversity and 
engage in cross Cultural dialogue. 



Sufficient understanding of scientific principles to 

permit comprehension of fundamental physical processes* 
awareness of the role of science and technology in 
contemporary life and recognition of the potentials and 

limitations of modern scientific methods. 



understanding of the role of religion in human 

life sufficient to enter mto the oeliefs and traditions o' 
others and engage m dialogue with them, recognize the 
interaction between religion and society, clarify and plac» 
in perspective Judeo -Chr i st i an tenets *no practices. 



• Understanding o< tne structure and functioning of t-- 

principal institutions of U.S. society, including governme"- . 
business and finance, the press, education, religion, the 
f ami ly . 



9' ^n understanding of human deings as developing 

individuals, ootn os rcro log ica 1 I y and physically, to 
facilitate holistic f^ealth and personal growth. 



9, AwAr«n«sm of onm*% o^n tal^ntSi potential! 



10. Knowl«dg» of som« field in depth* 



Appendix B: Sample Transcript and Analysis 



ERIC 



4i) 



Mars Hill Col lege 



The interviewer is Robert Melvin and I will 
be interviewing Dwayne who is communications 
major, Phillip who is recreation major, and 
Scott who is a management major. The first 
question that I would 1 ike for us to talk 
about is what you consider an educated person 
I think all of us will recognize and educated 
person when we meet them . What are soipe of 
the characteristics or marks of such a person 
would you say? 

I think knowing how to communicate and 
expressing yourself is real ly important. 

Okay , commun icat ion. 

And k now i ng your se If how to express your 
feelings- I don't really consider an 

educated person as one that can quote 
Shakespeare verbatim but I think a person 
that has a good knowledge themself in the 
world and what's going on today, to me is an 
educated person . 

and to be able to express yourself, the 
communications part is a major factor in that 
you think? Very good, Dwayne. Who would add 
to that? 

I think an educated person is somebody that 
thinks ratiofially about situations and tries 
to put prejudices aside as much as possible 
and not c lose your mind to certain issues and 
they're aware of other peoples beliefs and 
ideas and you have to accept them for what 
they are- You might not agree, but you have 
to accept other people's beliefs. And at the 
same time an educated person is more willing 
to learn , always wi 1 1 i ng and open to do 
things, to learn things. I don' t th ink it 
real ly had any th ing to do with an educat ion 
level- Some of the smartest people I've ever 
known, never even finished high school • 



It's an openness , and 
i mpor tant i ngred i ant « 

Wi 1 1 ing to learn. 



a tolerance, is an 
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Mars Hill Col lege 



Interviewer 



And especial ly knowing what you 
and willing to learn it- So 
important attitude. Very well put 



don't know 
that's an 
I think. 



I nter V iewee 



You do have to be open m i nded and have a 
realization that people do have different 
concepts and you have to take and know about 
those concepts to know what you want to do . 



I n t er v i ewer 



How close are you men to 
person would you say? 



being an educated 



I nter V i ewee 



Well I feel like you never cease getting an 
education out of life* even though someone 
may consider you an educated person and 
someone else might not. It all depends on 
wants of people- 



I nter V 1 ewer ; 



It's a process 



I nter V i ewee 



It's something that you will always have to 

one of things that's 
my life? in the five 
is being more open 



work on too. That's 
changed in the most in 
years I 've been here 
mi nded about stuff. 



I nter V iewer : 



I nter V iewee 



Now Scott has talked about his time at Mars 
Hill mak ing him more open to others more 
tolerant and accepting of others. How have 
your years at Mars Hill changc:d you to help 
you to become an educated person? What ideas 
would you share on that? 

I believe in having an open mind. I felt 
like before I came to Mars Hill, I was a lot 
more trusting, saw no evil in the world. But 
just getting out a little bit on my own, 
getting away from home, I seen that you can't 
trust everybody- YOu have to set limits* and 
you have to be educated about those limits. 



I nt er V i ewer 



Okay , so you 
were when you 



re more word 1 y w i se than you 
can here you think? 



I nter V iewee 



I'm a lot 
others. I 
around me. 



more open-minded , accept ing of 
m more open to what 's go ing on 




Mars Hi 11 Col lege 



I nter V iewer : 



that 
there 



been particular courses or 
you have had, that you feel 

person 
Were 



Now have there 
exper iences that , 

have helped you to become the kind of 
you consider an educated person 



particul ar courses that you 
that you think may have contributed 



have had 
to that . 



I nter V i ev^ee : 



More of the people that I 've gotten, from 
looking at him and talking to him and having 
classes and spending some time on campus is 
Dr . Lindberg is real ly open. If you see him 
he's always trying to learn more and he's 
also always open to new ideas. I mean he's 
not a closed minded person. I 've learned a 
lot from his classes and just being around 
him and seeing how he is, and that helps a 
lot • 



I nter V i ewer 



So you've had some out of class contact 
you ' ve me t at least one educ a ted per son . 



and 



I n ter V i ewee ; 



I guess being around a 
a t the schoo 1 . . • 



variety of people here 



I nter V i ewer : 
I nter V i ewee 



How would the rest of you respond on that? 

The social sci ence c 1 asses I fee 1 hel ped me 
out a great deal, like broaden your opinions 
and just see how dif"ferent people's cultures 
are around the wor 1 d - 



I nter V i ewee : 



Probab 1 y po 1 i t ica 1 sc ience helped 



understand 
dea It with 



me to 

what's going cn with issues being 
today . 



Interviewer 



Interviewee: 



Sometimes we hear a particular person 
referred to as a citizen of the world, or a 
world citizen. Thinking of yourself^ what 
have you learned here that would help you 
understand your place in the world better, 
and that you think may help you become a more 
ef f ec t i ve c i t i 2 en in the wor Id? Who wants to 
go first? 

In my opinion, in order to be an effective 
wo rid citi zen you have to have a concern for 
o thers i n order to try to put your ef f or t 
into mak ing this a better wo rid. You can't 
just be yourself because the world doesn't 
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Mars Hill Co 1 lege 



end with you. To be a world citizen you have 
to be concerned for others and just take up a 
special cause. I have not yet found what my 
cause is, but I have to feel like I can get 
out to the real world. 



Inter V iewer : 



I n ter v iewee : 



Okay , very good . 

Mine is about the same thing? just helping 
people and setting goals for yourself. Just 
trying to be of service to yourself i s what a 
wo rid citi 2 en i s • 



Inter v iewee : 



I think having a little bit more of an 
understanding for deciding what to do about 
itj under star. d i ng what the situation is? and 
how things got to be this way. Then make a 
decision as to what you think you ought to do 
about it. Having concern for others. 



Interviewer 



I nter V i ewee ; 



Inter V iewer : 



It requires some understanding of the rest of 
the wo rid too? does n't it? Have there been 
some courses that have helped you to 
understand the world better? do you think or 
help you to be a wo rid citizen? 

I feel with just my major in recreation, what 
that is providing for other people for their 
enjoyment , It goes a little bit beyond that 
to where you are providing for the people. 
My recreat ion courses being dealt toward 
social sciences? and those that... the 
invasion of therapautic reasons or whatever . 



Now what 
Hi 1 1? 



culture courses did you take at Mars 



I nterv iewee s 



Any of the social sciences? the philosophy 
c 1 asses ? taken f rench culture cl asses . 



I nter V iewer : 

Interviewee : 
Interv iewer : 
Interviewee : 



Okay, so french culture was the foreign 
culture. Has that increased your interest in 
what's going on in France? 

I can say yes, but I don't believe it has. 
Alright, an honest confession. 

Language and popular culture class, I 
understand more about why people do the 
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Mars Hill Co 1 lege 



things they dOf say the things they say. 
Dther courses I guess would be history* 
am&rican history. I learned even more than 
in high school i because the instructor made 
you want to learn. You enjoyed it- 

I 've got my history minor? I guess the 
american culture classesi the modern era 
classes dealt with Viet Nam and the latest 
i ssues . And then a 1 so I had two or three 
c 1 asses on Ch i na and Japan . 

Okay. Do you guys keep up with what's going 
on in eastern Europe just now in Lithuania* 
inside Russia? 

I probably don't know as much as I should. 

You are interested? Okay? good. When you 
came to Mars Hill, Scott? you were planning 
on a L)usines5 majors did you decide on the 
history major after you got here or had you 
intended to do that? 

After I got here I took a couple of classes, 
I got enough to get a minor. 

Okay, but that will always stand you in good 
stead as a person in business that you have 
this world understanding that an almost 
history major would provide you. That's 
i nterest i ng . What have you done outs ide of 
classes that have contributed to your 
understanding of the world and may have made 
you more nearly a world citizen. Have you 
had some outside of class experiences or 
activities? What would you say Dwayne? 

I jo i ned the f raterni ty and I learned more 
about getting along with others, just coping 
with others and their problems. I enjoyed 
doing the se?rvice projects and stuff. 

Okay, Philip. Have you had some outside 
exper iences? 

For me my outside experience was I cheered 
f or . . .watched the cheer leading squad for 
three years. That's helped me a great deal 
just getting out in front of people, and 



Mars Hill Col lege 



I n ter V i ewer : 



Interviewee: 



I n ter V i ewer : 



Interviewee: 



Interviewee: 



Interviewer : 



I nter V i ewee : 



feel ing comfortable with that. It's helped 
me make a lot of friends. I've learned that 
after being in sports in high school y through 
out the high school years f that I wanted 
something when I got to college* I felt that 
cheer leading was the way. The ones that are 
on the field are not always important* 
sometimes it's the peoplf* on the sidelines. 

If something were to happen in America in the 
mor n i ng that made Amer i ca env i r onmen ta 1 1 y 
unsafe to live in» where would you go to live 
in the world and why? Uhere do you think 
you ' d want to go ? 

I'd like to go to Australia, somewhere I 
guess you would call Americanized. There's 
still a little bit of country that you can 
walk around for a day or two and not see 
anybody . 

Okay* you'd like virgin territory, open 
spaces. That's interesting. YOu got to 
leave Dwayne, where are you going? 



My first choice would 
pictures I've seen in 
it. 



be Canada. From the 
books and read about 



I'd want to go to Australia too. I fig ^ if 
he goes and I qo too, everybody will be going 
too. I figure if something that disasterous 
happens somebody will have to stay home and 
clean it up, unless you want to spread it 
around everywhere e Ise . 

You just might stay here and clean up the 
environment, huh? Okay. Let me to ask you 
to think for a few minutes about the role of 
the artist in society. And we*re using the 
term artist in a very broad sense. The 
music i an, the author , the painter , the 
sculptor, any form of the arts. Why do you 
th ink ar t ist are important in a soc iety? 

My thoughts is that they are impcrtant 
because they keep your imagination growing. 
Without the artist our imagination would 
cease to be . They help us to form new 
opinions. We can look at a painting and see 
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I nter V iewer : 



I nter V i ewee : 



I n ter V i ewer 



Interv ie;vee: 



something that no one else has every seen. 
So the artist keeps our imagination going. 

Okay, that's an important idea. 

An artist I think challenges people. YOu 
just get a feeling from it tnat no else gets. 

Okay, it's an important idea. 

An artist I think challenges people? 
sometimes society doesn't agree with it but 
i t something you have to learn to accept • 
Some ar t i st may bring out things, or may 
reflect in his work what he considers ? \fe to 
be. People may not agree with that? but 
that's just his interpretation of what life 
can be. 



I nter V i ewer : 

Interviewee; 
I nterv i ewee : 



So you see the artist out on the fringes of 
society. Freedom is pretty important you 
think in that setting*? 

Yes . 

I agree with what they're saying. An author 
or an artist or photographer or whatever, 
they stimulate the thought process. They get 
the new ideas going, and then at the same 
time they reflect values that are present in 
the soc iety - One of these days peop le are 
going to look back at what these people did, 
and they're going to know kind of what it was 
like. 



I nterv iewer 



In a time of war, one of the very first 
things a nation does, is to hide it's art 
works, isn't it? Or if a fire breaks out in 
a building, the first thing they go for is 
the works of art. They try to save the works 
of art and let the building go if necessary. 
Why do you think that is? Why do you th ink 
that it's important to reserve the works of 
art? 



Interviewee; 



Interviewee; 



Probably because they think they're reserving 
their culture and their history. 

You also have got to be realistic, somebody 
has put a monetary value on those works, and 
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they're probably worth more than the 
bui Iding. 

There's the business mind at work. But 
you ' re r ight . 

It would be nice to think that's all that's 
beh ind it. There * sgot to be someth ing el se 
to it as far as the monetary value or 
whatever 9 or even sent i menta 1 va lue . 

And what you were just say i ng , preservi ng the 
culture. A part of our culture is lost if we 
lose this- What do you think freedom means? 
There ' s a 1 o t about freedom i n the papers 
now, with the break up of the communist block 
nations. What does freedom mean to you? Why 
do you value it? 

I guess it means being asle to think » and 
express your se If without having the fear of 
being persecuted for it. IF you have ideas 
that are different from somebody else's? 
you're always going to have to face the fact 
that they aren't going to agree with you. 
Somebody is going to look at you differently. 

That's a pretty precious gift, isn't it? To 
be ab le to be your sel f . Very good . 

I chink if they're not given a chance to be 
themselves, they feel like they 're c losed in. 
And if they don't have that chance to express 
themselves, then they're presenting a 1 ie to 
the world- 

I feel like freedom allows for change. To be 
pro or con for someth i ng , it may be a 
political issue, may be alright for a while 
but it starts to outgrow it's time. And with 
the freedom that we have in America, we have 
that choice to change it. I feel in eastern 
Europe they ' ve not al lowed freedom in the 
past as they do in our government, and their 
economy has just become stagnet, because it 
has not made the changes it has needed to 
make. So freedom helps out a great deal. 

That^s an important idea, don't you think? 
So that changes can be made gradually and as 
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they 're needed j rather than havi ng to keep 
them suppressed and then a revo lut ion comes 
that may be d i srup t i ve and destruc t i ve . 

Instead of using guns 9 use words . 

Very goodf very good. What would you mean 
say are the two or three most important 
experiences that you have had as students at 
Mars Hill College? They may be positive or 
negat i ve . We learn from bad experi ences as 
well as good experiences. They could be 
academic or social or spiritual or personal, 
cultural. What are some memories you're 
going to take away from here whether you want 
to or not? 

Probably the first day I got here, I just 
remember being unloaded, and just saying 
goodbye to my parents and watch them drive 
off and here I was- And there was absolutely 
no one I knew. I had to just kind of force 
myse If to get out and meet peop le - 

My feeling is the same as Dwayne's. When I 
left home.. -the day I ^eft home to come up 
here, I knew it was not going to be the same, 
I was leavi ng home. I was afraid of it, but 
I knew I had to accept it. Once I got here 
and got used to it, I realized that you can' t 
go through life being afraid. I've got 

strong with God being here. You have to put 
your trust in God, and accept things. That's 
one of the major things I believe I ' ve 
learned, and this has helped me insure where 
I can go out in the real world. 

I don't know what the biggest thing I've 
learned is. I think the biggest influence 
since I've been here is coach stevens, the 
head football coach. We're more friends than 
anything else. I don't look at him as a 
coach and I don't look at him as a p layer . 
The whole department, it's good to see people 
1 ike that can be successful . They're just an 
unbel ieveab le role model. I think that's 
probably the biggest influence that's happen 
to me since I've been here, is just seeing 
good people like that. Having them rub off 
on me some of their ideas. 
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It's good that that kind of experience can 
happen. Do you think it happens more to 
small schools like Mars Hill than the large 
school or does that make any d if f erence do 
you think? 

It probably happens more in a smal 1 school . 
Coach Stevens has had a bunch of players 
since I've been here. If he's had half the 
influence or even a little bit influence on 
them that he ' s had on me , he's done some 
thing Mell tgor th . • • nor th a lot. 

So football has been a positive influence in 
your life. Let's suppose that you're younger 
brother is going to be a freshman at Mars 
Hill next fal 1 f what do you feel you need to 
tell him that maybe nobody else would? How 
can negotiate Mars Hill and have a good 
enriching experience. What advice could you 
give him> that you think might would help? 
Have a go at it Ph i 1 . 

Well I have a brother here that's a 
sophomore. I really don't think there's any 
advice you can really give themi it's just 
someth i ng you have to exper ience . Once you 
leave home^ you can't have someone hold your 
hand everywhere you go . The exper ience is 
different for everyone. It might sound the 
same* but the feeling inside is different. 

So you tell him to go for it. Is that what 
you're saying? 

Yeah* just go experience it. 

Just deal with whatever comes along. I think 
it's going to be like any other thing he 
would go through. Just take the good with 
the bad. 

I'd probably say the biggest advice I would 
give him would be to not do what I did the 
first two years, and get more involved. I 
came to Mars Hill originally...! didn't come 
here for an education, I came to play 
football, and I lived football for 4 i/E 
years. Maybe not focused in on something so 
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I nter V i ewer : 



I n ter V i ewee 



I nter V i ewee ; 



I nter V i ewer 



I nter V i ewee : 



Interviewer : 



Intervi ewee: 



Interviewer : 



Interviewee: 



muchf and get involved 
not saying I regret it, 
toj but if I look back 



in more things. I'm 
I did what I wanted 
at it,*. 



And maybe Scott's brought up a question that 
my be good for the rest of us to respond to 
and that is what would you do differently if 
you had your years to go over again at Mars 
Hill. Scott says he would become involved in 
campus ac t i V i t i es ear 1 ier . What wou Id you do 
Dwayne? 



Probab ly . . . I can' t real ly think 
anything I regret. Just try to 
involved with the people. 



back on 
get more 



I would have 1 iked to have gotten more 

involved in the student or gan i za t i ons > and 

the student government . I just liked to have 
had just a little bit more control. 

Sco 1 1 ? you ' re ready . 

I really don't know. I think democracy is 
going to spread. I think that eventually 
eastern Europe is going to see that... I mean 
they're seeing it now. Communism doesn't 
work. I think there's going to be a lot more 
effort put toward doing something about the 
environment for the next ten years or so at 
leastp I mean they're realizing it now, but 
it's going to be even more important. It's 
going to start to hit home a little harder. 

Okay* so growth and democracy and greater 
environmental awareness are what you see 
happening over the next fifty years. Very 
good. Dwayne, what do you see? 

More unifying of the world's governments, 
people will start agreeing on one path to 
take. And probably a lot of diseases will be 
wiped out by cures. Probably come into play 
on the other side of that maybe more 
diseases, new diseases will come up. 



Okay, that's interesting 
and some new ones emerg. 



We conquer some 



I feel that instead of having a cold war 
11 
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between the east and west which is gradually 
going awayi wi 1 1 be a cold war between the 
nor th and the south • Such as south Amer ica 
and the mid east, North America and Europe. 
I don*t think there will be anymore troubles 
such as d iseases or whatever , but I don ' t 
think there will be any less as Dwight said. 
I feel like instead of having the green house 
effect 9 wh ich they are not even sure exist 
now. They 're having quest ions about tha t , 
they'll have other types of effects, I think 
everything will be the same but it will be 
different things. So we'll still have 
troubles, we'll conquer the troubles we have 
now, but we will just have new troubles to 
conquer . 

Interviewer: It will be an interesting fifty years anyhow. 

And you guys will see it. I'll see the first 
decade or two . Now let me ask you what you 
think is the most important thing that you 
1 earned dur i ng your freshman year at Mars 
Hill. It doesn't have to be just one thing. 
How do you think you changed dur i ng that 
first year? Tell us maybe how it happened 
that you learned what you learned. 

Interviewee; Probably becoming more independent. Deal 

with that and not having family or parents 
by - Because I remember comi ng home over 
break one time, and I'd got in the door and 
they would always ask me, well where are you 
going? And I would have to tell them, so I 
wasn ' t used to it. 

Interviewer: Okay, that's an important part of growing up 

isn't it? 

Interviewee: Coming from high school to college, I didn't 

feel I had that much more freedom, because I 
feel like my parents were pretty lenient on 
me when I was in high school, but I respect 
that and I always let them know what I was 
doing. I felt like that helped me whenever I 
came to college it wasn't really a shock to 
have everything, all the freedom that you do 
have in college. I did feel like the biggest 
thing that I learned that it's easy when your 
parents are'nt there when you turn around and 
ask them for help. You have to start doing 
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things on your own. AlsOi another thing that 
I learned is that life isn't always the way 
you want it. I left a girl friend back home 
that I thought that I was real ser ious about , 
came to find out that it wasn't meant to be 
that way . Li f e does have it's shor t curves 
and turns. So I believe that's what I 
learned the freshman year . 

Pretty interesting year i sounds to me like. 
Scott . 

Probably dealing with your time* learning how 
to structure your time and getting the most 

out of it. Probably more importantly* 
learning how to deal with different types of 

people. Learning how to deal with types of 

people that you hadn't been around before. 

Learning how to live with other people. 

What do you think is the most important 
question that you are now dealing with, in 
your own life? Question or issue? what's 
going on in your life right now, Phillip? 

Most important one for me was graduation 
within the nex t week or two . It is where am 
I going from here? It's not something I 
worry a great deal about. One of the things 
that I've learned here too, also, is that you 
can't worry yourself sick about everything in 
life. You need to have faith in a God, or 
faith in yourself, and work hard in whatever 
you do. You will succeed in something. I 
would just like to know what it is I'm going 
to be succeed i ng in. 

Part of the same thought is where I wi 1 1 be 
fifty years from now or where am I going to 
be just a year from now? What am I going 
have to do to obtain my goal s? 

Probably just decide what's going to make me 
happy . Dec iding what path I'm going to go 
down • 

Do you think you have gotten some help at 
Mars Hill College to help you deal with these 
questions that you're facing? 




Mars Hill College 

I*ve learned a lot? but I still don't know 
what I'm going to do yet> which way I'm 
go i ng . 

I feel that just going to school without just 
jumping straight out into to the working 
wo rid has helped me to mature a lot more than 
it would have jump ing in straight from high 
schoo 1 • Whether that be going to Mars Hill 
or going to some other school* I don't know 
if it made any d i f f erence - I'mglad I did 
come to Mars Hill, it's like a family 
tradition, I've had brothers and my mother 
and aunts come here before • So whether I ' ve 
gone here or somewhere else, I don't know 
whether it would have mattered . 

What do you think is the most important thing 
that you're going to take away from Mars 
Hill? That might not have been in your life 
if you had not come? 

Some of the people I've met . 

The peop le you ' ve met . 

Probab 1 y understand i ng and knowi ng all the 
peop le with different backgrounds. Different 
families* seeing how they all function and 
what makes people tick. 

My opinion is the same as Scotts, greater 
understanding of people. And also my 

friends* I've made several good close friends 
that I wi 1 1 have for the rest of my 1 ife. 
They will always be there if I ever need 
them- I can be there for them if they need 
me. The understanding of people and just 
seeing how they work with things and how I 
work with things. 

If the president of the college invited you 
into his office to gi ve him some of your 
parting advice on how to make Mars Hill a 
better college* what advice would you give? 

You've got your music majors and the other 
percentage of athletes* and those people are 
kind of separated. 
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Freshman Essay 

One goal of all liberal arts colleges in to enable students 
to express themselves well in writing. Another is to instill 
an appreciation of the liberal arts education and the broad 
perspect i ves it adds to the life of the individual. Al though 
it is difficult to assess a college's ef f ec t i ventss i n 
reach i ng these goals, this exercise is one way to approach 
it. 

Most colleges ask freshmen to write an essay as part of their 
English placement process. These essays can easily serve 
both purposes, and this would be ideal for several reasons. 
It avoids duplicating the effort and time required for 
faculty and students to schedule, write or administer the 
assessment • Us i ng the p I acement essay also more or less 
assures that it will be written at the very beginning of the 
freshman year, before students have participated in freshman 
composition* Most importantly, motivation for good 
performance is inherent in the situation. However, some 
colleges may have reasons to incorporate writing the essay 
into the normal Freshman English course or into a personal 
development or orientation class- 
Entering freshmen will be asked to write a short essay (no 
more than ^0-^5 minutes or 2 pages) on one of the following 
topics. 

1. Imagine that you are a senior at this college. How are 
you different than you are now? If you could choose one word 
to capture that difference, what would it be? Please 
elaborate on the ways it would apply. 

2. A group of other freshmen are standing around talking 
about why they chose this college and what they expect it to 
be like. Someone asks you what you expect to be most 
important to you in your college experience. What do you 
say? 

Before the essays are evaluated for placement purposes, they 
will be copied. Therefore, it is important to test your 
copier before instructions are given* and ask students to 
write with black ink or pencil that will copy adequately. 
One set should be used by the faculty for placement grading 
in whatever ways this is nor ma Illy conduc ted . If the Eng I i sh 
faculty estab 1 ish- and apply the writing criteria, it might be 
a logical place to maintain the graded placement copies, so 
that they can be compared later with the senior essays. 
The other set will be labelled and eventually stored where 
they can be retrieved when these students are seniors. In 



some colleges this could be in the students* advisor's 
portfolioj in others a central place may be designated for 
these and/or all assessment data. It is importantf for the 
sake of continuityj that at least the dean, the person 
responsible for assessment and the secretarial staff are all 
aware of the location and procedure for maintaining these 
f i les. 

Pit!?r yses thin the cg_l_l ege : 

Before the freshman essays are stored^ there are other ways 
the information can be used- Both student development staff 
and faculty could read the essays to help understand the 
or ientat ion of their new freshmen as they enter co 1 lege. 
They could also be useful for freshman advisors. 

Another approach would be to compile a list of the "words" 
your freshmen chose to write about i and group them to see 
whether there are themes or common sets of expectations among 
your students. The list might be distributed to freshmen 
through their freshman orientation courses? or used as a 
basis for discussion in the course- 

You might also want to see whether your word groups are 
related to other information? such as family background, 
major* Myers-Briggs types or whatever else you already have 
and think is relevant. If there is a relationship between 
those and the kinds of changes students anticipate* both 
students and faculty might benefit from knowing about it. 

Using the information right away will help give your 
assessment effort credibility. Both students and faculty will 
be more supportive of a process which they perceive as 
providing useful and relevant information. This is also one 
way to demonstrate that assessing student outcomes can be 
done in a non~threateni ng way. 

E^5D59II^ lyn? comparisons: 

The consortium colleges could share this kind of information 
if the assessment coordinator could compile a list of the 
words students choose to write about as capturing their 
antipated changes or the most impor tant aspec t of the i r 
education. Ue could group the words and see whether there 
'similariies and differences in the expectations of students 
by college. This information will be helpful in interpreting 
other kinds of data as well. 

Sttnior Essay 

During the second semester of their senior year* we will ask 
students to wr i te ano ther essay . The quest ions will be 
reworded s 1 i gh 1 1 y : 

1. Now tha^ you are a senior* how are you different than you 
were as a freshman? What one word would best capture that 
difference? Please elaborate, 
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2, From your perspective as a senior* if you were talking 
with a group of freshmen* what would you tell them has turned 
out to be the most important aspect of your college 
exper i ence? 

Assessment 

Seif ^5§f§sfnent: The process of thinking and writing about 
their educational experience as sen i or s and mak ing explicit 
for themselves how they have changed and what changes they 
value most is worthwhile in its own right. It becomes more 
valuable* however* when students can compare their current 
writing and ideas with their freshman idi?as and writing. For 
this reason it is helpful to copy the senior essays and 
return them and the freshman essay together. Students can 
then assess their own progress* both in wri^ ing and in 
relation to the content. 

Faculty assessment: Faculty will also evaluate both the 
changes in writing ability and the changes in the students 
resulting from their college experience- Probably the best 
way to evaluate the changes in writing would be to mix the 
freshman and senior essays together and grade, score* or 
evaluate them all according to the same criteria at the same 
time. This would give you a more consistent picture of the 
college's effectiveness in improving students' writing than 
scoring them at separate points in time. 

Data analysis: Students can be graded according to the 
holistic scoring system or according to some agreed-^upon 
criteria such as those I distributed earlier. If the 
scoring information is recorded in this kind of format* you 
can then make comparisons between majors* between those who 
took certain courses* etcetera* in terms of changes in 
writing proficiency. 

Student Name Sample 1 Sample 2 Sample 3 

scores 

cr i ter ion 1 
cr i ter ion 2 
cr i ter ion 3 



Retention : 

You can also use the unclaimed freshman essays as a part of 
assessing the retention situation and learning more about who 
drops out* compared with who stayed. Comparisons can be made 
between institution using this data as we 11. 

Other i 

It would also be useful to give the essays of the individuals 

being interviewed to the faculty members who will be 

conduct ing the interviews of those particular seniors* ei ther 



as a part of the basis for their discussion with the students 
or as a way of expanding their understanding of the 
interviews after the fact. 

Some of this information may be usef u 1 to the admissions 
staff and to the development staff » as well as to general 
education faculty. 
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MtMO 

To: Appdlachldn College Proqram Deans 

From: Alice Brown 

Re: Institute on Assessment 

Date: January 20, 1987 



Attached is a copy of an announcement which appears in the newest edition 
News and Notes , which will soon be sent to you for distribution to 
your facuTTy. The Institute ^n Assessment is designed to help you and/or your 
representatives select or develop aoproaches to evaluating the success of your 
efforts to provide quality educational experiences for your students. The 
institute is limited to a maximum of 20 participants. The faculty who will be 
teaching in the institute would prefer even a smaller number but have agreed 
to this maximum. Although the Exxon Education Foundation has provided seed 
money for a series of conferences or institutes^ the cost of this institute or the 
expenses associated with this institute are to be recovered through the registra- 
tion fee charged; and it will be difficult to recover those costs with fewer 
than 20 participants. The faculty would prefer that we have at least two 
participants from each of the campuses, and we are certainly not opposed to 
having three or four faculty or administrators from one campus. 

As the announcement indicates> the institute will be held in April and a 
follow-up session will be held in the fall. The S500 registration fee includes 
all sessions. The announcement includes information about rooms at rates that 
are quite good for the Lexington area. Those of you who anticipate having James 
Still or Pew Fellows on our campus this summer might be able to make arrangements 
to share an apartment or room with them for the two-week period of the institute. 
If you are interested in special rates which would allow three or four persons 
to share a room at the Sprinqs Inn, I shall be happy to get those rates for 
you. The cost of the institute will include coffee breaks and an occasional 
reception or meal. For most meals participants will be on their own. but there 
are a variety of cafeterias and fast food restaurants near the meeting place 
where food can be purchased at a nominal rate. 

The attached announcement lists tfie renis^ration deadline, but as soon as 
you know who you anticipate will attend, please let me know. If I can qive 
you any additional information about the institute please do not hesitate to 
contact me. I am anxious to have this institute prove successful; I would 
like to see it beLOiue a model for future collaborative efforts between the 
College of Education and the Appalachian CoHede Prodrain. As with all of our 
programs, this institute is designed to meet your expressed needs. I shall 
look forv/ard to hearing from you as we finalize our plans for the institute. 
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UNIVERSITY OF KENTUCKY 

APPALACHIAN COLLEGE PROGRAM • 64i south umestone • lexington. KeNiucKV 40506-0333 • PHONE teo^i 257 3746 

A pHvx.HAHi Of T»a Ar«*ALAc»4tAM CcNtcn 

February 5, 1987 



TO: Dr. R. Pfau Dr. B. Thacker 

Dr. A. Perkins Dr. D. Schmeltekopf 

Dr. D. Boldon 



FROM: Alice Brown 




RE: Assessment Institute 

Recently you received an announcement about the assessment 
institute. Deadline for registration was listed as March 15. Karen 
Carey, who will be visiting your campus soon, has recommended that 
we hold ten places for persons from your institutions. To assure that 
spaces are available for your representatives, I am requesting that 
you let me know by March 2 the names of those from your campus who will 
attend the institute so that they can be enrolled prior to our 
accepting others. 

During Karen's visits to your canpus you might want to discuss 
the institute with her and identify appropriate participants from your 
campus. As Dr. Weatherford suggested at our November meeting, both the 
FIPSE project and the institute will provide excellent opportunities for 
you to develop a plan for assessing the educational outcomes of your 
programs. I hope to hear from you soon. 



71 

T-nir^' AN EQUAL OPPORTUNITY UNIVtRSfTY 

hKJC 



UNIVERSITY OF KENTUCKY 



APPALACHIAN COLLEGE PROGRAM • t)4i -,t.iun iiMtsidNt • itntNi.ioN <-.cnux>-< 4o',.>«o!jj ' o-x.'H rbc*. ..'s/ J74t 



Dear Col league : 

I am writing to you about an Institute on Assessment to be held in 
Lexington, Kentucky, on June 7-20, 1987, at the University of Kentucky. This 
Institute will bring to faculty and/or staff from small, private colleges, the 
expertise necessary for designing and implementing assessment programs on their 
campuses. Specific points to be covered will include the defining of objectives 
and identifying their potential measures; selecting sources from which to 
gather information; choosing instruments; collecting, coding, storing, processing, 
and analyzing data, and reporting results in meaningful and useful terms. 

Faculty and consultants during the two-week period will include 
Leonard Baird, formerly of ETS and currently a professor in the Department of 
Educational Policy Studies and Evaluation at UK; Charles Elton and E. W. Kifer, 
also professors in that department; Jonathan Warren of Research in Higher 
Education, Berkeley, California; and others whose expertise will match the 
interests of participants. 

A planning meeting at the University of Kentucky on April 29 (noon-6 p.m.) 
for participants who can attend will focus on identifying specific needs and 
interests in the area of assessment. Those attending that meeting are expected 
to come prepared to discuss the types of information currently collected on 
their campuses and the information which their colleges would like to have 
made available. A brief follow-up meeting in the fall will attempt to 
determine how the Institute has or will benefit participants' colleges, 
and how the colleges miqht continue to work together to share information. 

The total cost of the Institute, including planning and follow-up 
sessions, is 5600. Participants will be responsible for meals and housing but 
rooms in campus dormitories and local motels are available at reasonable cost. 
Registration is limited to twenty. 

For further information, please contact me at the above address or 
phone number. 



April 2, 1987 



Sincerely, 




Alice W. Brown 
Di rector 



AWB/sea 
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COMPONENTS OF THE INSTITUTE ON ACADEMIC EVALUATION: 

This institute will be designed to provide you with the ideas and 
tools necessary to do the following: 

1. Identify and examine the values inherent in your programs, 
policies and procedures 

2. Fjormulate or clarify the specific objectives, goals and 
purposes of your college that are a result of the values 
identified in (1) 

3. Determine criteria for measuring success in meeting your 
objectives 

4. Defining, obtaining, analyzing and interpreting numerical 
data and other information that relate to your criteria 

5. Detennine and explain the extent of success or failure in 
meeting your objectives 

6. Specify the relationships between experiences in your college 
and the outcomes of your college (the impact of your college 
and it's programs) 

7. Identify unplanned and undesirable side effects 

8. Detennine the relative impact of your college and it's 
programs and the impact of external variables 

9. Recommend the alteration or replacement of features of your 
program 

10. Set up a continuing review of your program results 

11. Assess the value, benefits or social utility of your college 
and its programs. 

Although the new accreditation standards only go through (5) above, 
academic evaluation can serve many other useful purposes, described in (6) 
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to (11) above. That is, with only a marginal increase in effort, your college 
could obtain considerable information that would allow you to conduct 
continual self study and provide your college with information you could 
use in your day to day decisions. 

Some of the specific points to be considered in the institute include 
the staffing, timing, estimation of costs, and organization of tasks in the 
evaluation. We will also consider ways to define objectives and identify 
their potential measures, the sources the information will come from, the 
choice of specific instruments, data collection, coding, storing and 
processing the data, analyzing the data, and reporting the results in clear, 
meaningful and useful terms. 
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DATA ANALYTIC PERSPECTIVES AND TECHNIQUES 
Perspectives: 

1. Data analysis as detective work rather than 
sanctif ication ; 

2. Producing valid but understandable displays of data 
and results for different audiences; 

3. Disaggregating global measures while looking for 
patterns of results. 

Techniques: 

1. Exploratory Data Analysis (EDA) with an emphasis on 
understanding what is in the data set; 

2. Approaches to describing and displaying how 
students, faculty, courses and programs change; 

3. Dealing with messy data. 
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SUNDAY, JUNE 7 



6:00 p.m. 



TENTATIVE 

AGENDA 

INSTITUTE ON ASSESSMENT 
June 7^-20, 1987 

Taylor Education Building 
University of Kentucky 
Lexington, Ky 



Dinner and Welcome 
Springs Inn 
Main Dining Room 




^..^C^MW ^^^^^^ J 



MONDAY, JUNE 8 - WEDNESDAY, JUNE 17 



9:00-11:00 a.m. 



Leonard Baird 



11:00-4:00 p.m. 
6: 30 p.m* 



Skip Kifer 

Chuck Elton, University of Kentucky 
Student Learning Theories 



THURSDAY, JUNE 9 



7:00 p,m. 



Jim Rogers, Southern Association of Colleges 
and Schools 
SACS and Assessment 



WEDNESDAY, JUNE 10 



6:30 p.m. 



Beth Goldstein, University of Kentucky 
Ethnography 



THURSDAY, JUNE 11 



5:00 p.m. 
8:00 p.m. 



Reception tor Fellows and Institute Participants 
Theatre 

"Brighton Beach Memoii:s'* 



We have moved the reception to allow those of you who have expressed 
an interest in leaving Lexington for the weekend the opportunity to 
do so* Those who stay in town are invited to have dinner at 
Spindletop (our "old" faculty club) on Saturday evening. Sunday a 
tour of Shakertown, in-rluding a ride on the riverboat there, is 
available for those who wish.) 



FRIDAY, JUNE 12 



6:30 p.m. 



John Smart, VPl 

Retention Research 
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Page 2 

Tentative Agenda 
Institute on AssessLv^nt 



MONDAY. JUNE iS 



6:30 p-m- 



Margaret Jorgersen, SRLB 

Placement Standards and Remediation 



TUESDAY, JUNE 16 



6: 30 



Free Evening 



WEDNESDAY, JUNE 17 



3:00 -5:00 p.m. 



Jerry Lunney, Council of Kentucky Independent Colleges 
and Universities 

Disciplinary Views of Truth and Proof 



6 : 30 p*m* 



Jerry Lunney 



THURSDAY, JUNE 18 - FRIDAY, JUNE 19 



9:00-11:00 a.m. 



Leonard Baird 



1;00-4:00 p.m. 



Bud Warren 

Course Level Assessment Using Final Exams and 
Syllabi 



6: 30 p.m. 



Andy Grimes, University of Kentucky 
CP^a/itative Measures 



FRIDAY, JUNE 19 



9:00-11:00 a.m. 



Leonard Baird 



1:00-4:00 p.m. 



Bud Warren 
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PLANNING AN ASSESSMENT: 
SOME QUESTIONS TO CONSIDER 

A. What is the purpose and background of the assessment? 

1. What inputs, environmental factors, processes, or 
outcomes are to be assessed? 

2. What are the critical points at which evidence will 
be required for decisions? 

3. What rules, procedures, assumptions, and principles 
are involved in the decisions? 

4. Who will make decisions and what is the process by 
which these will be made? 

5. Does the overall situation suggest, require, or pro- 
hibit certain tactics and strategies? 

6. What timing considerations are involved? 

7. What are the limitations on costs? 

8. What are the specific assessment tasks? 

B. What information is to be collected? 

1. Are the particular pieces of information unambiguously 
defined and collectible by objective and reliable means? 

2. From where or from whom is the evidence to be collected? 

3. By whom is it to be collected? 

4. What instruments or procedures are to be used? 

5. Will the collection of evidence in itself seriously 
affect the input, environment, process or outcomes? 

6. Will the collection of evidence become a regular part 

of the process, or is it an add-on for a one-time assess- 
ment? 

7. What is the schedule for collection of information? 

C. What procedures will be used for organizing and analyzing data? 
1. In what form is information to be collected? 
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2. wm coding be required? If subjective judgments will 
b« required in coding, are the criteria for these ade- 
quate? Who wi11 do the coding? 

3. How win the data be stored, retrieved, and processed? 

4. What analytic procedures are to be used? 



Is the reporting procedure clear? 

1. Who will receive reports? 

2. Will reports be organized by analytic procedures, by type 
of data, or by decisions to be made? 

3. Will reports include the practical implications regarding 
the various possible decisions to be made or leave these 
implications for the project staff or administrators to 
ascertain? 

4. Is the assessor(s) to state explicitly the oarticular de- 
cisions which he (they) believe(s) are supported by the 
evidence? 

5. When and in what detail are reports to be made? 



How is the assessment to be assessed? 

1. Who will be involved -- project staff, the assessors, 
decision makers, some presumably more objective indi- 
vidual? 

2. What will the criteria used in this second-level assess- 
ment be -- costs, program improvement. Impact on further 
planning of related enterprises? 

3. To whom and when is this report to be presented? 

4. What decisions are to be anticipated as a result of the 
report? Will they include improvement of assessment process 
in the future? 
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The Sources 

Much but not all that we will talk about is covered in A pplications . Basics and 
Computing of Exploratory Data Analysis by Paul Velleman and David Hoaglin and 
publfshed by Duxbury Press of Boston in 1981. I will ask Alice to order 10 or so of 
them. The cost of the book is about $20. 

I will have applications of the techniques and, hopefully, you will have a data set 
where most of the techniques will be appropriate. 

Your Data Set 

This is not required but is desirable. I am going to use Placement Examinations as 
my example. It is meant to be an exemplar or model for what you will bring. 

1. What data you bring or what data you have were collected for a reason. One has 
an interest in the data because one has a question to answer. Take placement 
examinations. They are given usually for one of two reasons: 1) to put students 
into courses that are presumed to be hierarchically ordered. For instance, those 
who have good marks in mathematics and high placement scores take Mathematics 
102 while the lower scorers are placed in Mathematics 101; or, 2) to provide special 
instruction (often called remedial) for those who score very low. 

2. Suppose one had placement tests for the latter purpose, to provide remedial 
instruction. A rather obvious question is whether your remedial program is working. 
Let's suppose the program was designed to help students read and write better. 
Suppose further that you decide the program is working if students do well in 
Freshman English and stay in school. A data set like the following would help you 
answer your question. 

3. Choose one year (probably four years ago since you are interested in retention) 
and for each student assemble the following data: 

ACT or SAT Scores 
Placement Examination Scores 

Single item results from placement scores (not necessarily the whole 
test but lets say at a minimum 10-20 items) with the students raw 
responses to each question. 

An indication of what kind and how long the student was given 

tutoring or a remedial class. 

An indication of the content of the remediation 

Grades in Freshman English 

Whether or not the student was graduated. 

Miyor Field of study. 
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For a particular student the data could look like this: 

ACT - English 12 Mathematics 15 Science 14 Social Science 13 Composite 14. 

Placement Examination - 45 

First 15 items on test - 123451234512345 

Kind of remediation - 1 Tutoring 

Length - 2 semesters 

Content - 1 Writing 

Grade C 1st semester C 2nd semester 

Graduated 1 - yes 

Major - Mathematics 3 

Coded, your data for each student might take the form: 

ID# 12 15 14 13 14 45 12345 12345 12345 1 2 1 C C 1 03 

If you were to bring a data set of 50 cases as above that would easily be enough. 

4. The above is just an example. An adequate data set would include the followinu 

a. A notion that there was a question to be answered 

b. A program or set of experiences 

c. Some scores on a test 

d. Some item level responses 

e. A criteria - how would you know if the outcome was good 

f. Something that implies different groups - in the above case that variable is 
Major. 

5. In what form should you bring your data? 

a. On a floppy disk in a standard ASCII text format; 

b. On a tape; 

c. On punch cards; 

d. In a suitcase in which case vou would have to spend an evening or so putting 
them on a floppy or on the mainn-ame. 



Skip Kifer 



APPALACHIAN 



COLLEGE 



PROGRAM 



WORKSHOP 



LEONARD BAIRD 



DVFUCATBD BT tASBgGMAFBKS J77 SOVTB UMBSCIXfNB, LOUSCrON, KY 4RM0 (i»f) 2S1-4440 

84 



CONTENTS 



1. Assessment: The National Picture 

Peter Ewell "Assessment: What's It All About?" 

Terry Hartle "The Growing Interest In Measuring the 
Educational Achievement of College Students" 

2. Beginning With Educational Goals 

Howard Bowen "Goals: The Intended Outcomes of 
Higher Education" 

"Cognitive Learning" 

3. Clarifying Desired Educational Outcomes 

Oscar Lenning "The Outcomes Structure: Introduction, 
Overview, Potential Uses, and Guidelines for 
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Designing Alumni Studies 

Mike Stevenson* R. Dan Walleri and Saundra M. Japely 
"Designing Follow-up Studies of Graduates and 
Former Students" 



Defining. Obtaining, Analyzing and Interpreting Data and 
Other Information 

Jame^ Endo and Terry Bittner "Developing and Using 
a Longitudinal Data File: The University of 
Colorado Experience" 



Determining and Explaining the Extent of Success and Failure 

Charles J. McClain and Darrell Krueger "Using Outcomes 
Assessment: A Case Study in Institutional Change" 



Reporting and Using the Results of Assessments 

Mary Kinnick "Increasing the Use of Student outcomes 
Information" 

Robert D. Brown and Larry Braskamp "Summary: Common 
Themes and a Checklist" 




Junt_- , 1V117 
Appa 1 ach i aii Coll ege J- r cigr aiit 
Universal t y of f entuct y 
L e ^: 1 ng t o ri * 1^ Y 

On June 7-20, 1987, ein institute on Assessment was field 
at the University of Kentucky for facility and administrators 
from the 38 colleges participating in the University's 
Appalachian College Program. The two-weeli program was 
designed to al low the par t ic ipants to become i nvol ved i n 
an intensive study of approaches to evaluating institutional 
effectiveness. Eleven persons from seven different 
institutions attended; the maximum number that the institute 
faculty had recommended was ten. A list of participants is 
at tached . 

A preliminary session w^th the participants was held on 
April 29 in Lexington to discuss the types of data currently 
being collected on the small private campuses and the types 
of information which the colleges would like to have made 
available. The agenda < see attached) for the two-week 
session was planned as a result of discussions held to 
identify the specific needs and resources of the 
par t ic ipants • In June participants brought to the institute 
data sets from their campuses which could be.- used to generate 
results from various tests and computer programs which are or 
could be made available to the participants on their h^^me 
campuses. In April the participants were also given a 
suggested reading list and a copy of a publication of the 
office of Educational Research and Improvement entitled 

ll? B^S£19^J[} biStJ^L Education- The textbook for 
the institute was App I icat ionsf B^^LCS|_ and Qomayit i^ng of 

Attached is a list of the participants. The two leaders 
of the institute were Dr. Leonard Baird and Dr. Edward Kifer, 
Dr • Baird's degree is from the University of California 
(UCLA) and he had almost IS years of experience with 
Educational Testing Services <ETS) before joining the faculty 
at UK in 1983. Dr. Kifer'^s Ph.D. is from the University of 
Ch icago with a spec iality in measurement evaluat ion and 
statistical analysis; he has been at UK since 1972- 

Guett presenters included Margaret Jorgensen from the 
Southern Regional Education Board; Beth Goldsteivt (UK College 
of Education); Andrew Grimes (UK Department of Management in 
the College of Business); Jerry Lunney (Associate Director of 
the Council of Kentucky Independent Colleges and 
Universities); Charles Elton (UK College of Educcttion); John 
Smart ( Prof esst#r EduCcitional Statistics? Virtjinia 

Er|c 8 7 BEST copy AVAILABLE 
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Polytechnirai i r.t. t i title > ; J i c. .ji--. (Oi.v-etc. , bouthe.r. 
Association of Colleges and bcht cl-); r*nd Jc..,atr.«.. W.^, > en 
<a private consultant fron. Be>»eley. California). 

Participants spent eac-h morning v«or» » ..g wilt, data which 
they had brought froir. tJ.eir campuses, le.*r.iing the latest 
techniques for organizing and displa/i..g information. 
Afternoons were spent witti Dr. Baird, discussing the 
literature on college assessn.e.,t and current testing 
techniques. Evenings were reserved for cjutst speakers with 
the exception of one free evening and o..e evening dedicated 
to a social function where the participants were guests of 
the UK Theatre Department for a drama production. 

Topics covered included the purposes and background of 
assessment, the collection processes, procedures for 
organizing and analyzing, reporting and evaluating 
assessment techniques. Special presentations covered such 
topics as assessment in the classroom (as opposed to 
assessment as a general campus concern) and student learning 
theor i es . 

The verbal evaluation session held during lunch on the 
final day of the institute indicated that the group felt that 
the institute should be repeated in basically the same format 
as this year for a new group of lO to 15 faculty or 
administrators from the Appalachian colleges. A major 
benefit they found in the current institute was the sense ot 
collegiality which developed during the four weeks and the 
participants requested opportunities which will allow them to 
continue to work together while on their home campuses. A 
one-day follow-up session which was originally scheduled has 
been extended to be three days in November at the University 
of Kentucky. Jim Rogers of SACS will try to schedule one of 
the day-long workshops on assessment to be sponsored by SACtj 
to follow the meeting of the institute participants so that 
they can attend that meeting during their visit to Lexington. 
Several of the participants also mentioned that it would be 
helpful to have Dr. Baird and/or Dr. Kifer visit their 
campuses as consultants; they will try to arrange for such a 
consultation once they return to their homes. 

The participants also suggested that they should return 
to Lexington next summer for the final three days of the ne)!t 
institute (if we hold another) with the expectation that if 
new speakers are added to the agenda, they could be scheduled 
during that final three days. The participants also thought 
it would be helpful to share comments with new participants 
about how they had made ttse of the information acquired 
during this summer's session once they returned to tr.eir 
campuses . 

The College ••f td«-cation, which currently administers a 
grant from FIPSE to wor' -^i h the colleges on assessing their 
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re Id ted Xi* Ihdt \n yjn t * 

tifjCM i!bi«r e J b/ i%,t- \ I, (oilttijtr f'l ctm '.-4^-^^. ht-lil in 

the fall o( \'"^3^ L^int to* u*_i*(J imj letcOtKMi cniiil eitli itioji; ovei 
l^O eittenJed tlwii one d^/ cc^iif *'i etic e cost oi 

apfji ox i (fia ttrl / out:-- third ti<t/ <: t«it 'it the institute, h 
quest ior» vvhicfi educat*.r- ciddre^^'^ lepecxtedly noirx lelevcHi^t 

for the ColletjT- l't<.qr^ifi: i it hetter to give (1> a few 
people art intr-M:-.-* irrdt'tjih cdnc a i ioi i^:^ 1 e>:pei leitce, c«r (d) 
many people * (Miit- ^j* ntit-'l i i if or nii^i 1 1 or* wliicti ttopefully v«< i 1 1 
lead tfieiii t<,» -^^^e^ I'^it rotM t- i nf im lUc^ t a Oi i «m"i their owjii. 
The 198tj t i«n ^ r^i trM ic 4^ gert-i -ittd a fiio f it i«f cippi r»,. i ct»i^ te I y ^ KjOO 
after e^,'pe.r^t-i , In boiti ttie l9yS corifereoce and ttie I9ti7 
institute, fertility ti«'fii Itie colleger, served by the 
Appal acti I Colleijr- l-rcijiMiu tt^turned to their c^inpu^be^ with 
riew idoris aiid tj*-^t'tM] 1 1 1 f tt« iii^t t jon and techoiqt*es- The* inain 
question i *^ how ft»uctt of titt- i nfttrmdit i iMi and tecfnu quet=. were 
adopted on trie i cninpu^jt.-^ crnd ^irice tlie colleqe«i contributed 
*50(> per per^^on i ^'^^ w*t> 1 1 a., ttie co^t of meals and hou'iiiiq) it 
vMOuld seem mc-re iitely tticrit a<Jdition«l support for the 
participants to implement ->omt- af ttie suggested tectiniqueib 
would follow *f>ucr« a tnqh investment. A budget summary and 
several letters from par 1 1 c i pcr^n fcs refiecting the valt^e of ti'^e 
institute are altacfied- rU a meeting in Novembei of all of 
the deans from tt^e 3B colleges served by the Appalachian 
College Pi oqr am the quest i;«n of what t i nd of i restitute or 
conference should be scheduled for 1988 will be addrestsed. 

The Appalachian College Program orice again expresses 
its gratitude to Exxon for making it possible to prc^vide 
oppor turn t ies to the f acul ty and administrators of the 
private colleges in central Appalachia througli utilization of 
the resources of ttie Uriiveisity of Kentucly- 
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6:30 p.m. 



l.crJii^Ard Band 



Skip Kifet 



Chuck Elton, Univctbity ot Kentucky 
Student Learning Theoricb 



TliUHSUAY, JUNt: 9 



7:00 t>-nt. 



Jim Kucjcrs, Southern Association ot CoUc.jt i, 
and Schools 
SACS and Assessment 



WtiDNESDAY, JHNt: lU 



6:30 



heth Goldstein, University ut Kentucky 
Ethnography 



THUHSOAY, JUHL 11 



5:00 p.ro. 



8;00 p.m. 



Keceptiun for Fellows and Inbtitute Participants 



Theatre 

-Brighton Beach Memoiis* 



We havt^ moved the reception to allow those of you who have expressed 
an interest in leaving Lexington for the weekend the opportunity to 
do so. Thotic who stay in town are invited to have dinner at 
Spindlclop (our "old" taculty club) on Saturday evening. Sunday a 
tour of Shakertown, including a ride on the riverboat thcie, is 
dvailable tor thoiie who wish.) 
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PLANNING AN ASSESSMENT: 
SOME QUESTIONS TO CONSIDER 



A. What is the purpose and background of the assessment? 

1. What inputs, environmental factors, processes, or 
outcomes are to be assessed? 

2. What are the critical points at which evidence will 
be required for decisions? 

3. What rules, procedures, assumptions, and principles 
are involved in the decisions? 

4. Who will make decisions and what is the process by 
which these will be made? 

5. Does the overall situation suggest, require, or pro- 
hibit certain tactics and strategies? 

6. What timing considerations are involved? 

7. What are the limitations on costs? 

8. What are the specific assessment tasks? 

B. What information is to be collected? 

1. Are the particular pieces of information unambiguously 
defined and collectible by objective and reliable means? 

2. From where or from whom is the evidence to be collected? 

3. By whom is it to be collected? 

4. What instruments or procedures are to be used? 

5. Will the collection of evidence in itself seriously 
affect the input, environment, process or outcomes? 

6. Will the collection of evidence become a regular part 

of the process, or is it an add-on for a one-time assess- 
ment? 

7. What is the schedule for collection of information? 

C. What procedures will be used for organizing and analyzing data? 
1. In what form is information to be collected? 
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2. Will coding be required? If subjective judgments will 
be required in coding, are the criteria for these ade- 
quate? Who will do the coding? 

3. How will the data be stored, retrieved, and processed? 

4. What analytic procedures are to be used? 



Is the reporting procedure clear? 

1. Who will receive reports? 

2. Will reports be organized by analytic procedures, by type 
of data, or by decisions to be made? 

3. Will reports include the practical implications regarding 
the various possible decisions to be made or leave these 
implications for the project staff or administrators to 
ascertain? 

4. Is the assessor(s) to state explicitly the particular de- 
cisions which he (they) believe(s) are supported by the 
evidence? 

5. When and in what detail are reports to be made? 



How is the assessment to be assessed? 

1. Who will be involved project staff, the assessors, 
decision makers, some presumably more objective indi- 
vidual ? 

2- What will the criteria used in this second-level assess- 
ment be -- costs, program improvement, impact on further 
planning of related enterprises? 

3. To whom and when is this report to be presented? 

4. What decisions are to be anticipated as a result of the 
report? Will they include improvement of assessment processes 
in the future? 
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BUDGET SUMMARY 



The attached budget indicates that the total e><pendi tures for the 
Assessment Institute held June 7'20, 1987, at the University of Kentucky 
were $10,5^7*1'** Approximately 75 percent of this total was used for 
honoraria for speakers with the remainder being used to cover the cost 
of refreshments, books and materials, and the expenses of a coordinator 
for the two weeks • 

Each participant paid $500 for program costs, producing an income 
of $5500. In addition, those who elected to keep the textbooks provided for 
their use during the institute paid for those, a total of $168.00. 

Given the basic income and expenditure figures, the institute would 
appear to have resulted in a loss of $A879*1^* However, contributions of 
$4,125*05 from the College of Education, primarily for expenses related 
to speakers, brought the loss to only $754.09* This amount was covered 
by the grant from Exxon, leaving available for future conferences 
$9245.91, including grant funds and income from the conference. 

Deposits from participants were made to a new conference account 
(total deposi t'''$5668) . The expenditures for the conference, less those 
amounts paid by the College of Education, were taken from the grant account, 
leaving $3,577*91 in the grant account. Therefore, next year, when a conference 
or second institute is held, there will be a total of $9245*91 available from 
the two accounts. The $3,577-91 remaining in the grant account will be spent 
by September, I988. Hopefully, income from future conferences will allow 
that account to continue to provide funding for a series of conferences, 
with each conference replacing with Income that which Is spent for 
expenses. 

Since a major purpose of the grant (Strategies for Survival) was to 
provide "seed money'* to build a conference program for administrators and 
staff of private colleges in Appa^achla, that goal Is being met. 
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Dr. Alice W. Brown, Director 
Appalachian College Prograa 
6^1 South Linestone 
Lexington, Kentucky if0506-0333 

Dear Alice: 

I attended the Institute on Aesessment with some reluctance and 
mlsglTlngs because of numerous other summer commitments and plans* 
I came away from the institute impressed with its organization, 
breadth, and overall quality. Both you and Karen Carey are to be 
commended* 

The two principal professors, Drs* Baird and Kifer, were well-prepared, 
very knowledgeable, and extremely generous with their time, materials, 
and help* Probably the most outstanding guest lecturers were Andy 
Qriaes and Beth Goldstein. Ms. Goldstein gave us some very useful 
training In athnography for assessing the content of the senior 
interviews. I wish that we had had more than one session with her. 
Furthermore, it was helpful and important to hear Jim Rogers from 
the Southern Association of Colleges and Schools. As you have In- 
dicated, I expect that we will need follow-up Information and possibly 
a workshop on the SACS plans and materials promoting assessment as 
an integral part of accrediiation efforts* 

The pace of the institute was intensive; the sessions long.. Probably 
too much material was covered in the two-week period. In future in- 
stituteS| perhaps more time for reflection should be allowed to pro- 
mote better questions and more discussion. Also, towards the end of 
the Institute, one or two short meetings scheduled only for the par- 
ticipants might be beneficial. (We tended to continue the discussions 
at lunch and dinner but not always efficiently nor with everyone present.) 
Among many other things, I have learned that assessment Is a complex, 
multl faceted, and even controversial issue facing higher education. I 
still have a very considerable amount of material to digest and 
asslmulate from the Institute, not the least of which involves a 
wealth of bibliography. 

I appreciate the spirit of friendliness and the effort to make us 
comfortable which balanced favorably with the professional nature 
and structure of the institute. 

Sincerely yours, p 

Dorothy H. Schnare 

Ascistant Professor of English 
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July 3. 1987 



Alice W. Brown 
Director 

Appalachian College Hrogcam 
6^1 South Limestone 
Lexington, Kentucky A0506-0333 



Dear Alice: 

I wish to take this opportunity to thank you for putting together the 
excellent Institute on Assessment, June 8-19, 1987. Union College will 
clearly benefit from the information learned at this institute. Union will 
organize Us assessment effort beginning this fall, and it would have been 
difficult to prepare for this without having participated in the Institute. 
LeonaM Baird and Skip Kifer both provided excellent instruction. The 
specUl guest speakers were all very helpful. 1 am especially glad that you 
were able to bring in Jim Rogers of SACS and Jerry Lunney of CKICU. 

I hope that your program will be in a position to continue to disseminate 
information about assessment. 




Hubert P. van Tuyll 
Assistant Professor of History 
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Dr. Alice Brown. Director 
Appalachian College Program 
641 South Limestone 
University of Kentucky 
Lexington, KY 40506-0333 

Dear A I lce» 

Now that I have been back for a few days. I've had 
a chance to reflect on the Assessment Institute, and 
here are my thoughts about the experience. 

My genera) reaction Is overwhelmingly positive. I 
had been reading some articles on college assessment 
before I arrived, and I wasn't too sure If the 
Institute would add significantly to what I had been 
learning on my own. I was pleasantly surprised. Len 
Balrd was an encyclopedia of information on testing 
procedures and had a solid command ot the assessment 
literature. He was especially helpful In responding to 
the group's requests for specific Information on 
particular tests, often bringing in samples of the 
tests In question. 

I was also surprised, perhaps even more so, by 
Skip Klfer's portion of the Institute. After hearing 
him talk about "visual displays" of data at the April 
meeting, I had the impression that we would be learning 
to construct basic graphs and tables* something that 
would not have been new. By visual displays* however, 
he meant EOA, which I had never even heard of before 
the Institute, and I thought I had a passing knowledge 
of statistics. So learning these new techniques was of 
untold value to me, and will Influence not only the 
work on assessment that I will be doing, but also the 
way I te^ch statistical analysis in my experimental 
psychology course. 

The guest speakers were also quite interesting and 
added a rich texture to the Institute - as good as Len 
and Skip were. It was refreshing to hear from new 
faces. I was especially impressed with Beth Goldstein. 
Andy Grimes, and Bud Warren. As a suggestion for next 
ttme, you might consider trying to get Ted Harchese. I 
heard him talk at a CIC meeting last November, and he 
was excel lent . 
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members of thi= =...fT.ir.er = group m order to check out our 
pr ogr e s t . 

Of part.r.,lar ir.tcr^^t to mo among the presenters were Dr. 
Beth Goldstein, ^r . Bud iJarren, and Dr. And/ urimes. Their 
focus on some of th.: qualitati-'e aspects of assestff.ent 
opened man>' new door = for me, and confirmed m> beliet that 
knowing often can go far be>ond quantitative measurement, 
impressive as that ma,* be. 
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In summary then, the institute was of real value 
to me. It taught me things that I did not know. It 
clarified Issues that had been obscure, and it gave me 
a solid base of information from which to develop an 
assessment plan for Wheeling. Finally, and perhaps 
most Importantly in the long run, I now have a group of 
colleagues with whom I can consult on assessment 
problems as they arise. 

I hope this feedback Is useful to you, and I would 
like to thank you again for all your efforts in 
organizing the Institute. Feel free to use my name in 
recommending the Institute to people in the future. 



SI ncerel y , 




Jim Goodwin 

Department of Psychology 
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UNIVERSITY OF KENTUCKY COLLEGE OF EDUCATION 



MEMO 

TO; Institute Participants 

FROM: Karen Carey R^*^ 
Alice Brown Alp 

RE: Meeting November 15-16 

DATE: October 21, 1987 



On November 15-16, the deans of your colleges have been 
invited to attend the annual gatherings of deans of the 
Appalachian College Program. We hope you will be able to 
attend the follow-up session of our Institute on Assessment 
to be held during those same two days. 

We have planned for you to meet with Drs. Kifer and Baird 
beginning at 2:00 on Sunday and ending at noon on Monday. 
The emphasis on Sunday will be on applications of computers 
to assessment data. On Monday emphasis will be on further 
exploration of assessment questions. 

Rooms at the Springs Inn will be held until November 1. You 
should contact the Springs directly to make reservations by 
calling the Springs Inn at: 

1-800-354-9503 - out-of-state 
1-800-432-0775 - in Kentucky 

Identify yourself as being with the Appalachian College 
deans meeting. Rooms are $46/night for single occupancy or 
for as many as four people per room (two double beds) . You 
can pay the Springs at check-out. 

We assume that you would prefer to make your own dinner and 
breakfast arrangements as a group so that you can continue 
your conversations informally. A lunch at the Faculty Club 
on campus has been scheduled for 12:30 Monday. The luncheon 
speaker will be Paul Eakin, who will talk about computer 
networks in Appalachia. If you would like to join the deans 
for that luncheon, please send $8 to Pat Smith at the 
Appalachian College Program. 

You should soon receive an invitation from the Southern 
Association (SACS) to attend a workshop on the new institu- 
tional effectiveness criteria for accreditation to be held 
here in Lexington on November 17 from about 9 a.m. to 4 p.m. 
You may want to make reservations at the Springs for two 
nights so that you can attend that workshop. £lsas& Is^ MS. 
know whether you plan to attend by calling Karen at 606-257- 
1202. Leave a message on the machine if no one answers. 
If the phone rings for a long time with no answer, it means 
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the line is busy, so please try again. If there are spe- 
cific problems or issues you would like to discuss or you 
would like Skip or Len to address, let us know that as well. 
If you have data that you want to have on the maiiiframe, 
please send it to Dr. Kifer ahead of time. 

We'll look forward to seeing you again on the 15th. Thanks! 



Ids 
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Appalachian College Assessment Consortium 

CONFERENCE ON COURSE-LEVEL ASSESSMENT 
OF STUDENT LEARNING 

October 25-26, 1987 

Maryville College 
Maryville, Tennessee 



berea carson-newman emory&henry ferrum lindsey wilson 
mars hill maryville mllligan Shenandoah union Virginia 
intermont warren wilson wheeling berea carson-newman 
emery&henry ferrum lindsey wlllson mars hill maryville 
mllligan shenandoah union Virginia intermont warren 
Wilson wheeling berea carson-newman emery&henry ferrum 
lindsey wilson mars hill maryville mllligan shenandoah 
union Virginia intermont warren wilson wheeling berea 



Funded In part by the Fund for the Improvement of Post Secondary Education 
(FIPSE) of the United States Department of Education 
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Sunday Afternoon 

3.30 - 5:00 REGISTRATION and RECEPTION 

Profntt Dining Room, Pearson Hall 

5:00 OPENING SESSION 

Welcome 

Dean Bolden, Academic Vice President 
Maryville College 

Greetings 
Dr. Richard Ferrin, President 
Maryville College 

Conference Overview 
Karen Carey, Consortium Director 

Issues in Course-Level Assessment 
Jonathon 'Bud* Warren 
Research in Higher Learning 
Berkeley, California 

6:30 DINMER 

7:30 - 9:30 COURSE GROUPS 



Monday Morning 

7:00- 8:30 BREAKFAST (Pearsons Cafeteria) 

8:45 - 10:00 GENERAL SESSION 
Bud Warren 

BREAK 

10:15 - 12:00 COURSE GROUPS 

12:00 - 1:00 LUNCH (Pearsons Cafeteria) 

Monday Afternoon 

1:15 • 3:00 GENERAL SESSION 



Special thanks to Dr. Sally Jacob, Psychology, and Mrs. Jane Huddleston, Administrative 
Assistant, Maryville College and Ms. Debra Grodln. Conference Assistant, University of 
Kentucky 
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AGENDA 

Appalachian College Assessment Consortium 
Johnson City^ Tennessee 

Garden Plaza Hotel Board Room 
February 1988 

Thursday 

3:00-^:30 Review of Consortium Purposes and Approaches 



^ I 30-6 1 00 Consortium Needs 

Institutional Questions and Concerns 



6:00-7:30 Dinner in Hotel Restaurant 



7i30 - 10 Data Needs and Questions 

Academic Prof i le 
Pace 

I nter V lews 
Essays 



Fr iday 

8:00 - 9:30 Continuation Proposal 
10:00 - 11:30 Evaluation and Dissemination 
1 1 : 30 Summary 
12:00 Lunch 
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Conference on Aesthetics in General Education 
Appalachian College Assessment Consortium 

SUNDAY SEPTEMBER 25 

11-1:15 Arrival and Registration 

1:30-2:30 Keynote Session Cardinal Room 

"Understanding Artistic Meanings Aesthetic Education 
versus Art Education** 

Phi 1 A 1 person f Department of Phi losophy 
Un i vers i t y of Lou i sv i 1 1 e 

2:30-3:30 The Arts in the Context of General Educations Shared 
Visions 

Faculty from three colleges share their college's 
perspectives on aesthetic education 

Neil Di Teresa? Berea College Art Department 
Julie Fortneyi Mars Hill College Music Department 
Rex Stephenson, Ferrum College Drama Department 

Refreshment Break 

3:^5-^:^5 Small Group Discussions — 

Room Discussion Leader 

Cardinal Room A P^ggy Hypes 

Cardinal Room B Peter Crow 

Maple Room Betty Stroud 

Dogwood Room Rebecca Watson 

Cardinal Room Alcove 
Participants will work in groups of about 8 people from 
various disciplines and from other colleges* 

^: ^5*5: 15 General Session Cardinal Room 

Di nner 

7:00-8:00 Uhat are the structures and practices Me use to 

translate our goals and visions into student learning? 

Short presentations from three colleges with different 
approaches 

Rosita SandSf Berea College Music Department 

Richard Mull in, Wheeling College Philosophy Department 

Robert Bonham, Maryville College* Music Department 

8:00-9:30 Small Groups — Discussion Areas 

9:30--10:00 General Session — Cardinal Room 
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MONDAY 



till 9:^5 Breakfast 



9:00-10:00 General Session — Cardinal Room 

HoH well is it Norking? Beginning to assess our 
effectiveness — some possibilities and examples. 

Leonard Bairdt Professor of Higher Education* 

Uni versi ty of Kentucky 
Karen Careyf Consortium Director, University of 
Kentuc k y 



10:00-10:15 Coffee break 



10:30-11:^5 Small Groups Discussion areas 



llt'^S-lSrlS Benerf^i Session — Cardinal Room 



Lunch 



1:15-5:15 Synthesis and plans for the future 

Oppor tun i t i es for cooper at ion between institutions 



This conference is supported in part with funds from the 
U,S. Department of Education^ Fund for the Improvement of Post 
Secondary Educat ion. 
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Appalachian CoXl«9« Assessment Consortium 

Conference on 
Assessing General Education 
Focus on Soienoe and Aesthetics Outcomes 
Pipestem State Park 
Pipestem, West Virginia 
May 23-25, 1989 

Aesthetics Section, Tuesday, May 23 

11:00 " noon Registration 
12:00 - 1:00 Lunch 

1:15 - 1:45 Conference Charge and Overview — Karen Carey 

1:45 - 2:45 General Arts Session: At our meeting last fall, 

the arts faculty articulated different ways their 
curricula were structured to provide an 
understanding of 

1. the process of creating 

2. various art forms, styles, structures, media 
and their importance 

3. aesthetic understanding and ability to 
articulate social and personal meanings of art 

4. awareness of the historical context of the 
arts 

Panelists Pat Verhulst, Mars Hill; Graham Paul, 
Warren Wilson, and Roslta Sands, Berea, will 
discuss approaches to assessment of general 
education objectives for the arts in each of these 
areas. 

2:45 - 3:00 Break 

3:15 - 4:30 Participants will meet in small discussion groups 

with faculty from other colleges to share and/or 
identify and discuss the processes and instruments 
which can best be used to assess the arts 
objectives addressed by the panel? 

5:30 - 6:45 Dinner 

7:00 - 8:00 Keynote Speaker — ^ Dr. Patricia Kerr, Center for 

Teaching and Learning, University of North 
Dakota* 

"On the Mature of Knowledge Construction In the 
Arts and Solenoss" 

8:00 - 8:30 Responses, questions and answers 

8:30 - 9:30 Small group discussion, mixed arts and science 

faculty tj respond to speaker's challenges 
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Wednesday, May 24 



7:00 - 8:15 Breakfast 

8:30 - 9:45 How can we use assessment data to improve general 

education? What information would cause us to 
make changes in the way we teach? 

1. Arts content 

2. Arts creation/process 

3. Arts in relation to social questions, science 
and technology 

4. Art appreciation in personal lives 
9:45 - 10:00 Coffee break 

10:00 ' 11:45 Small groups, arts faculty only, discussing above 

questions 

12 : 00 Lunch 

1:00 - 2:00 Informal meetings with colleagues from your home 

institution to arrive at some consensus about what 
are the best assessment approaches for your 
college and why. 

2:00 - 5:00 Park activities on your avn 
5:30 - 6:45 Dinner 

7:00 - 8:00 Aesthetics groups, report from each institution 

8:00 - 9:00 Group leaders meet to plan presentations for 

morning general session 

9:00 -10:00 Morning presenters meet 
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Aesthetics Section 

Thursday, May 25 



7:00 - 8:00 Breakfast 

8*00 - 9:00 General Session 

Presentations from Science and Arts Groups, 20-30 
minutes each on: 

Science: 

1. What should the outcomes of the general 
education science curriculum be, and how is it 
related to the overall general education program? 

2. How should they be assessed? How can the 
colleges help each other? 

Arts : 

1. What should the outcomes of the general 
education aesthetics curriculum be and how does 
that relate to the goals of general education? 

2. How should they be assessed? How can the 
colleges help each other? 

9:00 -10:15 Groups react and respond to general session. 

Change the composition of disciplinary groups to 
dots and non-dots. 

Each science group is to develop a response to 
the art groups' 8 o'clock presentation and select 
a spokesperson. 

Each of the art groups is to develop a response to 
the science groups' 8 o'clock presentation and 
select a spokesperson. 

10:00 - 11:30 Presentation of responses and discusc;ion. 
11:30 - 12:00 Evaluation and Planning 



12:00 Lunch and Departure 

This conference was funded in part by the Fund for the 
Improvement of Post Secondary Education, U.S. Department of 
Education 
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Appalachian Collttge Assessnent Consortliim 

Conference on 
Assessing General Education 
Focus on Science and Aesthetics outcomes 

Pipestein State Park 
Pipestem, West Virginia 
May 23-25, 1989 

Science Section 

Tuesday, May 23 

11:00 - noon Registration 

12:00 - 1:00 Lunch 

1:15 - 1:45 Conference Charge and Overview — Karen Carey 

1:45 - 2:45 Discussion groups with faculty from other colleges 

A. What are the objectives of the general 
education curriculum regarding science at your 
institution? 

BRING TO THIS SESSION: 

1) Copies of the goals for General Education at 
your college and/or a statement of general 
education science goals. 

2) Copies of the syllabus for a general education 
science course taught at your college 

2:45 - 3:00 Break 

3:00-4:30 B. How do you and your college go about trying to 

achieve the goals you described? 

5:30 - 6:45 Dinner 

6:30 - 7:30 Keynote Session: Introduction by Dr. Woodward 

Bousquet, Acting Dean, Warren Wilson College 

Speaker — Dr. Patricia Kerr, Center for Teaching 
and Learning, University of North Dakota 
"On the Nature of Knowledge construction in the 
Arts and Sciences** 

8:00 - 8:30 Responses, questions and answers 

8:30 - 9:30 Small group discussion, mixed arts and science 

faculty 
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Science Section, Wednesday, May 24 

7:00 - 8:15 Breakfast 

8:30 - 9:45 Panel: How can we tell how well we are achieving 

our goals regarding science and general education? 

1. Assessing understanding of science content 

2. Assessing the nature and processes of science 

3. Assessing understanding of the relationships 
among science, technology and society 

4. Assessing students* ability to evaluate the 
scientific information which affects their 
personal choices. 

Panelists: Jin Bier, Ferrun; Mary Ann Ghosal, 
Berea, L ^e Swensen , W arren Wilson , Frank Quick, 
Mars Hill" 



9:45 - 10:00 Coffee break 

10:00 - 11:45 Small groups, science faculty only, discussing 

above questions 

12 : 00 Lunch 

1:00 - 2:00 Informal meetings with colleagues from your home 

institution to arrive at some consensus about what 
are the best assessment approaches for your 
college and why. 

2:00 - 5:00 Park Activities 
5:30 - 6:45 Dinner 

7:00 - 8:00 Science groups report from each institution 

8:00 - 9:00 Group leaders meet to plan presentations for 

morning general session 

9:00 -10:00 Morning presenters meet 
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Thursday, May 25 



7:00 - 8:00 Breakfast 

8:00 - 9:00 General Session 

Presentations from Science and Arts Groups 

Science: 

1. What should be the outcomes of the general 
education science curriculum, and how is it 
related to the overall general education program? 

2. How should they be assessed? 
Arts: 

1. What should the outcomes of the general 
education aesthetics curriculum be, and how does 
that relate to the goals of general education? 

2. How should they be assessed? 

9:00 -10:15 Groups react and respond to General Session. 

Each science group will develop a response to the 
art groups* 8 o'clock presentation and select a 
spokesperson . 

Each of the art group is to develop a response to 
the science groups* 8 o'clock presentation a 
select a spokesperson. 

10:00 - 11:30 Presentation of responses and discussion 

11:30 - 12:00 Evaluation and planning 

Noon Lunch and Departure 

This conference was funded in part by the Fund for the 
Improvement of Post Secondary Education, U.S. Department of 
Education. 
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Pipestem Conference on Assessing General Education 
Arts and Sciences faculty, May 25f 1989 

Confmrmncm EvaluatiorM 

Terrific! Many thanks to you for doing all this. I t^ ink 
that if arts and Sciences people hadn't been together it i%ould 
have been less rich. Initially thought there should have been 
more interaction between the two groups. Now I'm not so sure? 
but it was terrific having Rick in the arts group. Could that 
kind of representation be structured in? Did science groups have 
anyone similar? 

Really interested in pursuing interchange of doings with 
arts and science folks. Also* anything you can do to facilitate 
i nter-campus visits iAiould be great . < We ' re go i ng to go bac k and 
get swamped • ) 

Please set up a way for this group to get back together { if 
you think it worthwhile). Thank you so muchi Karen. 

Graham Paul, Theater, Warren Wilson College 



a, I enjoyed the relaxed atmosphere and the low-key attitude of 
Karen Carey. The conference was well organized and the speaker 
was very good. There were good introductions and discussions, 
and the end result was much stronger than I expected. 

Suggestions have group leaders that are able to keep 
directed focus and accurate time schedule. 

Nancy Hicks, Education, Virginia Intermont 

3. I feel this experience has been of considerable value to me 
since I was at step 1 in knowing about this formal program of 
assessment as required by SACS. I was pleased to discover that I 
have been doing many of these assessments in my classes in an 
informal way . 

^. I felt that the early sessions were spent primarily in a re-- 
hash of what our task was last Fall, i.e., acquainting each other 
with the goals and objectives and structures of the arts courses 
at our respective institutions. I'm sure that was quite useful, 
but I was expecting to jump right into the problem of how to 
assess the goals/objectives that we* for example* at Berea« have 
already formulated and thought through . I guess what I 
discovered was that some institutions and faculties are still 
grappling with what their goals ar ? and should be. And some 
institutions are still uncertain or uiinappy about even a basic 
mission statement. I guess this all makes me appreciate Berea- 
- a place with such a clear sense of what the school is all 
about, what its mission is, and what i ts par t icular focus is. 

Once we started talking about assessment! I learned a couple 
of new ideas for assessing aesthetic outcomes* and the discussion 
surrounding assessment means was very helpful to me. I leave the 
conference with the following conclusions: 
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— Assessment methods/means at various colleges may have 
some similarities! but for the most parti will have to be locally 
designed and particular to an in institution. This is due to the 
fact that while we all have some of the same basic goals* we have 
difference ideas and methods of attaining these goalsi e*g.f do 
we view content as the endi or content as the means toward an 
end? 

— Assessment will need to take p lace on a var iety of 
levels* For Berea* this should be at the individual course 
level f core course level* general education program level* and 
departmental level. This all seems very cumbersome and a little 
impractical. The logistics of all this is a little troublesome. 
Not impossible* but a lot of work. 

The conference has been great. The opportunity to interact 
with other arts faculty and to witness and 5hare their 
stragglings with content issues* methodology questions* etc.* has 
been a very rewarding experience for me* personally. Although we 
are sti I 1 quite far from any real consensus on some issues such 
as methods* and I am not sure that consensus is a desirable 
attainable goal* I felt heartened* particularly after the 
Thursday Morning joint session* that people are changing their 
views and are moving toward viewpoints and beliefs about the arts 
that are more encompassing and less restrictive. I was also 
pleased with the flexibility in the schedule* in terms of time* 
at this conference. Having some recreational time was great - 

Rosita Sands* Music* Berea 

5. I have mixed emotions about the values of this conference to 
me and my institution. I enjoyed the interplay with colleagues 
of other institutions and yet I felt that I and my institution 
had reached a level that I became teacher and sharer rather than 
learner and taker. I prefer to do the latter when I attend a 
conference. This is not to say that I didn't learn. The 
presentation and interplay with Pat Kerr was invaluable. The 
sharing of the articulation of problems and solutions with a 
different group — the arts — was enlightening. And we were 
glad to be able to share with others what we have already learned 
through the consortium. 

Maybe a "subconf erence" based on your knowledge of where 
each institution is in its assessment of knowledge would be more 
helpful. This conference obviously helped those who are just 
starting out ! 

6. The conference has been most helpful to me in 

1. giving me information concerning the entire area of 

assessment : present si tuat ion* process* problems* etc . 

2- giving me contacts with persons and college this seems to 

be potentially extremely valuable as I work with the actual 

process of assessment at Warren Wilson College 

3. stimulating thinking for me personally. I found a good 

bit of the discussion deeply stimulating* 

Pipestem is i«oncl«rful — always meet here! It is so 
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beautiful and the accommodations quite comfortable* 

I liked the mix between good planning and specific 
structure! on the one hand* and flexibility and ability to make 
adjustments in schedule* etc.* on the other. Good job* Karen! 
Good keynote speaker ! 

Thank you for letting me come? I enjoyed it thoroughly. 

7. Meeting schedule — I would suggest arrival in the evening 
with business meeting beginning 8:30--9;00 the following morning. 
Meet all that day and part of the evening. Adjourn at lunch time 
the following day- 

8. Attempt to get at the main point of the conference earlier 
in the meeting. I felt there was a lot of floundering around 
before the main purpose was attacked. The relaxation time was 
very important to help clear the thinking and rest the bottom. I 
learned a lot by the time we finished. 

9. Very helpful to me in ideas in lower level courses to 
generate student interest instead of students just sitting there 
as "jack-humps." The conference perhaps could be shorter because 
I feel everyone is talked out but they wont stop. I knew nothing 
atout what assessment was about before the conference. I would 
have benefitted from a preliminary introduction to Southern 
Association development of assessment criteria. It was good 
that the conference brought out strong points and weak points of 
my institution's curriculum compared with others. 

10. #Very useful and helpful eKchafigc of ideas!!! I felt some 
people had closed their minds and believe they have goals* 
etcetera worked out which will prevent the cyclic nature of 
taking assessment results and rethinking goals and objectives. 

Good ideas on methods of assessment but many will be 
difficult to carry out. 

Once assessment tools are in place — need to have another 
workshop on evaluation of these tools* and how they are or could 
be used« 

Two comments: 1. The problems presented here ( which is why 
we are developing goals and assessment procedures) need to be 
approached at an earlier level — secondary school systems* etc. 

S . Our goa 1 s and assessment means must be kep t s i mp le to 
prevent them from being lost in the paper work. 

11. For me* this conference was complicated by my lack of 
knowledge concerning: What is assessment? Why do we have to do 
it? What do we have to assess? etc. The conference served to 
help me address some of these questions* but would have been 
facilitated by some more introductory and or background 
information. Speaker more oriented to assessment may have been 
usef u 1 . 

I liked the relaxed* open-ended format of the meeting. 
Especially the way that* within this format* Karen held our 
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•nose*' to the^grindstone, " 

Personally* I feel the real value of this conference was to 
crystallize for me what assessment ''means." I also see« with 
some concern, that my institution is really deficient in this 
area and I think there is very little chance that I personally 
will be able to change anything at my institution, at least in a 
reasonable time. I feel that I need to be more knowledgeable 
about assessment and I need a lot more colleagues in other 
disciplines to be knowledgeable and aware of the necessity of 
assessment to make a difference at my institution. 

This was a very useful conference. It served to broaden my 
perspective of the purpose of general education in general as 
well as what assessment Mans* 

I think Jim's idea to use others as guinea pigs for 
classroom techniques is a great idea!!! I too, would like to 
continue the dialogue initiated here. I like Ed's idea of some 
sort of visiting program of conferences and faculty development, 
where we could go to their schools to see the results of their 
assessment program. I like Pat's idea of an aesthetics and 
science faculty show, where we can share classroom techniques and 
guided projects and labs, 

12. There have been several benefits of this three*day 
conference; Most important: an opportunity to share ideas about 
educational goals and assessment techniques. Seeing the work of 
Ferrum College* which is at least a year ahead of our school* has 
been very helpful in leading me to understand my goals as a 
teacher and the goals of Mars Hill College as an institution 

Rosita Sands said "I have been heartened."*' and I agree, we 
realize that we already know some valid assessment techniques* 
and that we are already using some of them* and can use others 
wi thout much troub le . 

Just as helpful: a chance to share teaching methods and 
experiences* a realization and affirmation of shared goals: 
meeting people I like* arranging teacher exchangeSf agreeing to 
swap poetry and criticisms of poetry with one of the conferees. 
All of this will enrich and help revive both my teaching and my 
wr i t ing . 

Commendations to Karen Carey for excellent planning and 
administration of the conference. Don't worry about punctuality 
(or lack of it caused by our total involvement in small group 
tasks) too much — everything worked out well. I would suggest: 
stop earlier. Aftei — dinner sessions were fruitful and well 
focused* but faces were grey and drawn by 8:^5. 

We rarely travel and confer in interdisc ipl inary groups . 
T^e chance to do this was a good idea: To discover shared goals 
and interests* just to gain better understanding and affection 
for colleagues. 

Good planning — flexibility in execution — for me* that's 
a good balance — good food and lodging* the wonderful view* 
birds* sunsets* tram ride — all are bonuses: But these 

experiences help restore a teacher's soul* which is* after all* 
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what she uses to teach with. 

Suggestion: an arts/sciences general ed festival in which we 
demonstrate what we do. Not the syllabi t but the performances 
and the projectSf might be shared. Thanks Karen, I loved being 
here . 



13. This conference was very good at making me more aware of the 
common nature of the arts and sciences. The ability to interacts 
react f and act as groups and individuals through the structure of 
the conference and to work both as isolated and interactive 
disciplines succeeded in distinguishing and joining the arts and 
sciences in my mind. This will certainly affect how I approach 
teaching and assessing students and evaluating my methods. 

Ulith regard to assessment analysis and induct ion« the 
conference was not as successful but still very effective. I was 
able to reasonably evaluate most of the methods we have chosen at 
Ferrum. Given the groping we have done here, I think we have 
chosen a good mix of techniques (relative to our excruciatingly 
developed objectives). Ue needed in this conference to have a 
session involved with critically evaluating specific evaluation 
methods and their programs of education. 

In general 9 more time was needed in all "Sessions and all 
sessions should be more tightly timed. Despite timing problemSf 
however* much was accomplished. Similar conferences mixing 
disciplines in other ways would be a very good idea. 

1^, Ue have met the assessment instrument and it is us. Very 
good workshop. 

I had no pr ior exper ience per se with assessment but am 
coming to understand it pretty well* Also got some good tips to 
improve my class. I feel a bit frustrated that I don't have 
more specifics to recommend to the Dean when I get backf but I 
may be able to do better than I think I can. 

Ken Morton, Carson-Newman 

15. I found this very interesting. I really liked the 
interactions with peers in the sciences and the arts. I have 
learned a lot and been inspired, amused and frustrated (on 
occasion) so the conference can be said to have "worked" for me. 
Specific highlights for me included 

1. The three^step paradigm which came Ued a.m. from Larry 
Stern at Mars Hill; 

Have students look at the course in terms of a problem or 
situation and answer: 

a. How important is this to me and society? 

b. Uhat kinds of questions do we need to ask about this 
issue? 

c. Where do we go to get information about it? 

S. Ken Morton' s observat ion that his k ids got more 
interested in chemistry when he slowed down in covering content 
and went into more depth about cellular research. 
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3. The open fire hydrant analogy for knowledge--onl y teaching 
^, The Wed • evening group (much angstf but good process* and 

pretty fair final product toot I think — we were the group 

represented by Jim Bier.) 

I enjoyed it a lot. Thanks &. happy birthday. 

Mary Ann Ghosal ♦ Berea 



16. This conference allowed me to learn A LOT from my colleagues 
on occasion. I would prefer that my division had "sent" m© and 
expected me to report back to them — How do you relate 
assessment back to curr iculum and co-cur r iculum : I f our 
colleagues don't have some ownership early we'll get resistance 
to incorporation of results into curriculum. 

Jim Bier, Ferrum 

17. We have come a long way since our initial conference in 
September. This time there was much sharper focus on what w© 
vgere assessing and ways of doing it. Clearly it is useful to 
continue to share experiences as we develop our assessment 
programs. The dynamic of arts people and scientists meeting 
together was excellent and stimulated imaginative thinking. It 
also produced a profound appreciation and understanding of what 
each area does and how it does it, Furthermoref I am inspired by 
the awareness that in our consort ium of smal 1 col leges* there is 
a wealth of talented* dedicated and concerned teachers. The 
conference has developed strong mutual respect and has g 1 ven me a 
very posi t ive feel ing that we are doing something r ight in our 
small colleges* and that we are on the right track in trying to 
make undergraduate education all that it should be for the 
students of our t ime • 

Joe Carter » Ferrum 

18. I came to this conference after working for more than a year 
on a mission statement* General Education objectives and 
assessment possibilities for our school. I use the word "I" 
loosely* for we as a faculty accepted the task the with guidance 
of a '^steering committee" and sub-committees. I must say in a 
very complimentary way that this conference very much tied up 
loose ends for me for that year of work* I appreciate very much 
the structure of the conference and the scope of material we were 
challenged to address. I found the exchange between my peers in 
the arts to be invaluable as we approach this formidable task of 
assessment. At the same time to interact with Science faculty 
and curriculum (often considered cold and narrow) and find so 
many similarities existing between us was refreshing and warm. I 
appreciate and support vigorously the results from both groups 
which emerged from the three days and think we all go back to 
our larger educational bodies armed with worthwhile leadership as 
we approach assessment as heralded by SACS. 

Providing the challenging speaker at the beginning was a 
definite plus* as was the afternoon of free time exploration and 
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the fun time at The Oaks. Opposing anyone who might have been 
bothered by the reality that inability to stick right to the time 
table provided* I found it invigorating that as groups we become 
so embroiled in our task that time become of less importance. 

I especially liked our final combined meeting where we could 
exchange our ideas and impressions. 

Finally, to you* Karenf thank you for your hard workf your 
congenial yet challenging personality and ultimately for this 
f i ne oppor tunx ty . 

Bev Thornton 

18. Karen - 

1. Institutional interaction this time moved far beyond show 
and tell. We exchanged ideas about course content (in relation 
to general ed goals). 

5. Now we need to design assessment instruments (or first, 
look at what already exists — get the Pace* Academic Profile, 
NTEf etc. — and evaluate these for our purposes. 

3. I believe we need to work in our respective discipline 
with an assessment expert. Test design is a skill that few arts 
faculty have had any updated information on — Please help us*! 

^.We could volunteer to bring examples of the assessment 
instruments we came up with and then look at what we broughtf 
i.e., 

1. Each institution brings a real portfolio and we look 
at i t as an assessment instrument. 

2, Bring a "senior letter' or a sel f -assessment sample 
and see what it is that we could assess with this kind 
of i ns trumen t • 

5. We might use the grid in the ''Learning to Learn** and apply 
it to assessment: 

1. We need some knowledge base in test design - a ROTE 
learning, if you will. 

2. We need to be GUIDED in this discovery of test 
design and assessment which is not course embedded. 

3. Then we can be AUTONOMOUS in designing institution- 
spec i f ic i nst rumen ts . 

Thanks for the article* by tr.a way. I enjoyed these two days. 

Thanks. Julie F. 

19. This has been an excitingi enlightening, creative experience 
for me. I came here with much frustration, schizoid 
understanding, and no real understanding or enthusiasm for 
assessment . 

I came to realize that assessment is a dynamic process and 
must occur at many levels. In fact, that feedback is assessment, 
and is the key to good or effective teaching • I now even agree 
that curriculum and program assessment at both the division land 
college-wide levels are necessary for education to remain viable 
within the communi ty . 

I return to Ferrum with some enthusiasm for assessment, with 
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some tools to approach that assessments and with some changes in 
outlook about my own teaching. Curriculum is not a static 
territorial thing* but a dynamic living changing organism which 
should be changed on the basis of our goals as educators and our 
own assessment of its effectiveness. Thank you and Alice Brown 
for this experience. I would like to see it grow and continue to 
be supported by the consortiu<o. 

Ed Thornton, Math-Science Division 

20. Comments on Conference 

As a relative virgin in the assessment process^ I found that 
this meet ing was enl ightening in several aspects . I must admit a 
past preference for cont^>nt -heavy general educat ion courses $ and 
while I still feel that content is important* I have been moving 
closer towards what I bel ieve will culminate in responsible 
teaching. Assessment techniques within the course seem to 
provide an extremely valuable technique for allowing us to gauge 
student comprehension. Within the sciences it is apparent that 
our general education courses must be directed towards dynamic 
and changing contemporary topics* but with a realization that 
scientific methodology should be fused to this approach. Ue 
should be cautious presenting "abstract" recommendations to our 
peers which are developed out of this conference (i.e.* we must 
present our recommendations in a tangible form which may be 
useful in the institutional assessment process.) 

Those of use who plan to use "in course" assessment 
techniques in our courses . next year should provide feedback to 
the consortium concerning our successes and failures. Feedback 
concerning colleague reaction to our suggestions should be 
gauged. I suggest Vancouver Island as our next meeting site. 

Ron Rosen* Berea 

21. I don't usually do conference evaluations because they ask 
such inane questions* but given Carey's Cosmic version* I do have 
some comments. First of all* thank you* Karen* for all this good 
work . I could be 1 1 ke some students and say 1 1 was good because 
I liked it* but this conference was very helpful in several ways: 

1. It gave me a new appreciation for general education as an 
important part of our missions* including a deeper understanding 
of interdisciplinary nature of knowledge. Ue really can learn 
something new after all these years! 

5. I feel very proud that we are working cooperatively 
rather than competitively* and think that it might help the 
public understand better what we do. It also made me understand 
the issues in assessment better* so I will be able to talk with 
more confidence to my colleagues in the fall as we get into 
assessment planning. Me have been interviewing our seniors* but 
not everyone is really involved in it. 

3. Meeting other interesting people who have the same kinds 
of objectives and face the same problems we do has been exciting 
and consoling* I have some new ideas from them and I also feel 
we have offered some ideas they can use. 
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^. Most of all f I have renewed by own sense of mission. I 
can make a difference, both for the college and my students. 

CN 

22. Not sure I know everything to help with assessment when we 
get back to our collegef but this helped a lot with asking 
better questions. The conference was a real surprise. I didn't 
know what to expect i but this definitely went beyond it. Great 
exchange of ioeas, conversations* both in and out of the 
sessions. Pat Kerr was a good addition. Loved the location. 

23. Encore! The conference was terrific and the sunset was too. 
We don't often get to discuss ou^ ^eal educational philosophies 
in a way that makes a good theor " ^ :cal . *hanks. 

JW (?) 
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Aopalachian College Assessment Consortium 



Conference on Assess inq Genera 1 Educat ion : 
i;he contributions of Science and Religion 

September 2^ and 85 t 1989 

AGENDA 

Sunday* September 24 

tl:00 — 1:00 Registration* Lobby 

1:00 — 2:15 Cardinal Room 

Introduction and We?lcome — Karen Carey 

The challenge of assessing general education 

R'^SQonses 

2: 15 — 8:30 Break 

2:30 3:^5 Structured Conversa t ions * separate Science 

and Religion Groups 

K Ulhat can you share from your college's 
general education program or your teaching 
experience that helps students deal with 
these issues? 

3:^5 5;30 Conversation with < olleagues on the 

grounds* weathfjr permitting 

5:30 — 6:^5 Dinner — Bluestone Dining Room 

7:00 — 8:00 A. Science faculty: Why should it matter to 

scientists whether religion is taught well? 



•-What would we expect to be the outcomes of 

general education courses in religion? 

-How will we know whether we have succeeded? 

B. Religion faculty: Why should the teaching 
of science matter to religion faculty? 

-What might we expect general education 
science courses to do for our students? 
-*How will we I'now whether vie have succeeded? 



Develop statements for morning 
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7:00 



Mondav September a5» l'?8*? 

Breakfast. Dissemination Planning Group 

meet ing 

8=30 - ',30 Whole group. Presentations from evening 

PesDonse 

Mixed science and religion ^^^^^^^ -Hv^nces, 
3. HOW do you evaluate scientific -^'^^^l^^ 
ideas? HOW do vou evaluate religious .deas? 

10:15 - 10:^5 Break - AND Some evidence to consider 

Implications for teaching and ^^^'^'^^"'3 
10:^5 - ^. What are the ways of Knowing involved? 

Back to assessment: how can we assess our own 
effectiveness, document needed improvements 
and put them into practice? 



11 s^5 — 12:30 



Lunch and departure 
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Appalachian College Assessment Consortium 



Part ic ipants 
Conference on Assessing General Education 
Science and Religion 

Nancy Hicks* Education 
Carson-Newman College 

Cindy Huf f i Nursing 
Carson-Newman College 

Carolyn Blevins* Religion 
Carson-Newman College 

Terry Weaver, Education 
Carson-Newman College 

Juiia Richards, President's Office 
Warren Wilson College 

Virginia McKinley, Language & Intercu 1 tural Studies 
Warren Wilson College 

Don Collins, Physics 
Uarren Wilson College 

Warren De Arment , English, Dean 
Shenandoah College and Conservatory 

Faye Wood, Religion 
Ferrum Col lege 

Jerry Sumney* Religion 
Ferrum College 

Jaci- Corvin, Religion 
Ferrum College * 

Ed Tho'-nton, Physics 
Ferrum Col lege 

James Bieri Chemistry 
Ferrum Co 1 lege 

Rick Williams, Agriculture 
Ferrum College 

Bob Swanson, Natural Science 
Union College 



Hubert van Tuyl I » History 

Union College l^S 



Michael McCoy* Religion 
Union College 

John Gasiorowski* Social Science 
Wheeling Jesuit College 

Roberta Meehan, Biology 
Wheeling Jesuit College 

David Hammona* Theology 
Wheeling Jesuit College 

Joseph Hayden, S.J.» Psychology 
Wheeling Jesuit College 

Ellison Jenkins, Religion 
Mars Hill Co 1 lege 

David Knisely, History 
Mars Hi 1 1 Col lege 

Frank Quick, Biology 
Mars Hill Col lege 

Larry Stern, Political Science 
Mars Hi 1 1 Col lege 

Sam Boggess 

Mars Hill Co 1 lege 

Dr. Garv N. Garner, Dean 
Bluefield College 

Clorf?nce Fouchee* Biology 
v^irginia Intermont College 

Daohne Haynes- Physical Science 
Virginia Intermont College 

Rebecca Watson* Sociologyi Dean 
Virginia Intermont College 

Alice Brown 

University of Kentucky 

Patricia Smith 
University of Kentucky 

Karen Carey 

University of Kentucky 
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Carl Bahner, Chemistry 
Bluefield CoUege 

Eugene Chaffin, Physics 
Bluefield College 

David Armbrister, History 
Bluefield College 



AppaUchlAn Coll«0» A«s«««««nt Conmortium 
Project D««ns' MM ting 
T«nt«tlv« AgvndA 

3 p.m. Ch»ck In 

3il3 Conf«r»nc« Room 

Introductions of n»w p«opl« 

Alic«t Brl«f history of th« Appalachian CoU»ga Program 
Kar«nt Bri»f history of th« Ass«ssm»nt Projsct 

and accomplishments. 

Currsnt status 
Alica: Currant orqaniiat ional status 

3:00 Dinnar 

6t30 Confaranca Room 

yhat Mould wa lika to do in futura? 
Program diractions and configuration? 
Costs* Financial Commitmant? 

T^»«riav. July 18. 198? 
8i00 Braakfast 

8t30 Confaranca Roon ^ „ w 

1. How should wa implamant daclslons mada Monday 

avaning? 

2. Fall confaranca plans 

10:00 Braak 

I. What can wa racommand to other ACP coliagas- 

8. Dissemination plans 

3. Continuation process 

12:00 Lunch 
1:00 Urap-up 



Participants 
Spencer MacWilliams* \4»rrmn Uilson 
Don Schmeltekopf * Mars Hill 
Mike Carter* Carson-Newman 
Clark Bryan* Carson-Newman 
Rebecca Uatson* vi 
Alice Brown* UK 



•Dean Bolden* naryville 
Joe Carter* Ferrum 
Jack MacOonaldf Uhaeling 

Pete Moore* Union 
Gary Weadmant Milligan 
Karen Carey* UK 



Not attending 
A I Perkins* Berea 
Warren DeArment* Shenandoah 



Rick Pfau. Emory & Henry 
Doug Boyce* King 
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APPALACHIAN COLLEGE ASSESSMENT CONSORTIUH 

ASSESSING STUDENT DEVELOPMENT 
AS AN 

ASPECT OF GENERAL EDUCATION 

APRIL 8-9, 1990 
PIPESTEM, UEST VIRGINIA 

AGENDA 

Sunday » April 8 

> lliOO — 1:00 Registration and Lunch (on your own) 

1;00 — 1:15 Conference Charge and Overview 

David Knisley and Stacey Street 

ijig — 2:00 Defining Development-From Theory to Practice 

Dr. Anand Dyal-Chand» 
Vice President for Student Affairs, 
Per rum College 

aiOO — 2:13 Setting Goals 

David Knisley, ACAC Director 

gjl3 — 3,30 Uorklng Groups on Student Development Goals 

existing goals-what are they 
ideal goals-Mhat should they be 
refine goals to assessable statements 
list programs and activities designed 
to achieve each goal 

3,30 — 3j<»3 Break and time to review display of group work 

3j<>g — ^:00 Overview of Assessment 

David Knisley 

t^tOO — 3:00 Working Groups on Assessing Goals 

assessing the achievement of the following 
common goalst leadership development* service 
fflotlvatloni values development, affective 
development, cognitive development 

g,00 — 6i30 Reflection and enjoying the scenery 

^,30 — 7t30 Dinner and assignments for Monday morning 

e,00 — Interest Group Discussions 

ERIC 



nond«y. Apr I 19 

7.30- 8:30 Br-.Uf..t <on your 

L Pr«»«nt«tlon of A«««»«" 



10*00-10:13 Br.*k A«.e»»«»«nt Instruments 

-I T*bl« Discussion oT h»»« 



David Knlsley 
Stacy w. S*^«^* 

Adjourn and Check-out 
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